North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Non-ISP companies multi-homing?

  • From: Howard C. Berkowitz
  • Date: Thu Jul 24 12:37:34 1997

At 19:09 -0400 7/23/97, netops wrote:
>Does anyone know of any non-ISP companies that have decided to
>multi-home? Is this a major trend for non-ISP companies running
>mission-critical applications on the Internet?
>
>So far, I only know of a couple, with PointCast being one of them.
>
>Thanks,
>Lincoln Silver
>FlyCast Communications

Let me try to answer this somewhat broadly, and still stay within the NANOG
charter.  I believe it is an operational issue to consider any
organization, whether ISP or not, that can affect internet routing.  An
enterprise that generates BGP announcements can affect such routing just as
well as a service provider.

A simple answer:  yes, I see some form of complex Internet connectivity
(I'm avoiding the term multi-homing due to ambiguity) in large enterprises
(Fortune 500 roughly) and in enterprises for which Internet access is
mission-critical, regardless of their overall size.

Mergers and acquisitions, where the joined enterprises each had their own
Internet access, often mean complex connectivity, at least for a transition
period.  Consolidation of separate divisional networks also creates this
situation.  I've found a very frequent case to be where a large enterprise
decides that Internet access should be available corporate-wide, but their
research labs have had Internet access for years -- and it works, as
opposed to the new corporate connection that at best is untried.

Different people use terms such as "multi-homing" in different ways.  Let
me propose a taxonomy I use in teaching design classes.  I have had real
networks that used all of these cases.

1. Single-homed.  Enterprise generally does not have ASN; all its
advertisements are made through its ISP.   Uses default routes to the ISP.
The customer is primarily concerned with protecting against link or router
failures, rather than failures in the ISP routing system.

   1.1  Single-homed, single-link.   A single path to the ISP.
   1.2  Single-homed, balanced link.  Multiple parallel paths from a single
customer router to an router.  Protects against link failures.  The single
customer router constraint allows this router to do round-robin
packet-level load balancing across the multiple links, for resiliency and
possibly additional bandwidth.
   1.3  Single-homed, multi-link.  Separate paths from multiple customer
routers to multiple ISP routers at different POPs.  Default routes
generated at each of the customer gateways are injected into the enterprise
routing system, and the combination internal and external metrics are
considered by internal routers in selecting the external gateway.  [I
generally recommend this to a customer who wants resiliency but wishes to
avoid the complexity of BGP]
   Special Case 1.1+, 1.2+, 1.3+.  While the customer is still
single-homed, an AS upstream from the ISP has a routing policy that makes
it necessary to distinguish routes originating in the customer from those
originating in the ISP.  In such cases, the enterprise may need to run BGP,
or have the ISP run it on its behalf, to generate advertisements of the
needed specificity.

2. Multi-homed.  Enterprise connects to more than one ISP.  May or may not
use BGP.  Wishes to protect against problems in the ISP routing system, and
will accept additional complexity and router requirements to get this.  May
also have differing service agreements for Internet access for different
divisions.

   2.1  Multi-homed, primary/backup, single link.  The enterprise connects
to two or more ISPs from a single router, but has a strict policy that only
one ISP at a time will be used for default.  In an OSPF environment, this
would be done by advertising defaults to both ISPs, but with different Type
2 external metrics.  The primary ISP would have the lower metric.  BGP is
not necessary in this case.  This easily can be extended to multi-link.
   2.2  Multi-homed, differing internal policies.  Assume OSPF interior
routing.  The main default for the enterprise comes from one or more ASBRs
in Area 0, all routing to the same ISP. One or more organizations brought
into the corporate network have pre-existing Internet access agreements
with an ISP other than the corporate ISP, and wish to continue using this
for their "divisional" Internet access. [I've seen this most often when a
corporation decides to have general Internet access, but its research arm
has long had its own Internet connectivity.  Mergers and acquisitions also
produce this case.]  In this situation, an additional ASBR(s) are placed in
the OSPF areas associated with the special-case, and this ASBR advertises
default.  Filters at the Area Border Router block the divisional ASBR's
default from being advertised into Area 0, and the corporate default from
being advertised into the division.
   2.3  Multi-homed, "load shared" with primary/backup.  [Thanks to Paul
Ferguson for the distinction between load balancing and load sharing.]
While there still is a primary/backup policy, there is an attempt to make
active use of both the primary and backup providers.  The enterprise runs
BGP, but does not take full Internet routing.  It takes partial routing
from the backup provider, and prefers the backup provider path for
destinations in the backup provider's AS, and perhaps directly connected to
that AS.  For all other destinations, the primary provider is the preferred
default.  A less preferred default is defined to the second ISP, but this
default is advertised generally only if connectivity is lost to the primary
ISP.
   2.4  Multi-homed, global routing aware.  Multiple customer router
receive a full routing table, and, using appropriate filtering and
aggregation, advertise different destinations (i.e., not just default)
internally.  This requires BGP, and, unless dealing with a limited number
of special cases, requires significantly more resources inside the
organization.

3.  Transit. While we usually think of this in terms of ISPs, some
enterprises may provide Internet connectivity to strategic partners.  They
do not offer Internet connectivity on a general basis.

3.1  Full iBGP mesh.  Connectivity and performance requirements are such
that a full iBGP mesh is practical.

3.2  Scalable IBGP required.  The limits of iBGP full mesh have been
reached, and confederations, route reflectors, etc., are needed for growth.


--------

Howard Berkowitz
PSC International/Protocol Interface (Cisco/Bay/Digital training partners)
  -- personal opinions, and other appropriate disclaimers