North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

2006.06.07 NANOG-NOTES Issues with IPv6 multihoming

  • From: Matthew Petach
  • Date: Fri Jun 09 07:51:02 2006
  • Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta;; h=received:message-id:date:from:sender:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; b=dQdqgwSBzY1eRkXXpC9THVkqo1l8gfS0uYMqc92IFboSyk9aIpcU0Zc2gO/yDsl001rOaKpwfSfjw2WEFr+KEIvOkz3jGY3Xui+abxRtS/BHgaFRj6lE8mD4/a1OKyysnKcIrPcRMORaq0mnFZq8KYwa3/LHN+CuUJDIoH2uANw=

(hope the inclusion of URLs in the notes isn't
making them all end up in people's spam
folders... --Matt)

2006.06.07 Vince Fuller, from Cisco
and Jason Schiller from UUnet

[slides are at:

IPv6 issues routing and multihoming
scalability with respect to routing issues.

how we got where we are today
define "locator" "endpoint-id" and their functions

Explain why these concepts matter, why this
separation is a good thing

understand that v4 and v6 mingle these functions,
and why it matters

recognized exponential growth - late 1980s
CLNS as IP replacement dec 1990 IETF
OSI, TP4 over CLNS--edict handed down from IETF
revolt against that, IP won
ROAD group and the "three trucks" 1991-1992
running out of "class-B" network numbers
explosive growth of "default-free" routing table
eventual exhaustion of 32-bit address space
two efforts -- short-term vs long-term
More at "the long and winding ROAD"
Supernetting and CIDR 1992-1993

Two efforts to fix it; CIDR, short term effort,
long term effort became IPv6.

IETF ipng solicitation RFC 1550 Dec 1993
Direction and technical criteria for ipng choice
RFC1719 and RFC 1726, Dec 1994
proliferation of proposals
TUBA == IP over CLNS
NIMROD==how to deal with it from high level

Lots of flaming back and forth, not much good
technical work.
choice eventually made on political choices, not
 technical merit.
Things lost in compromise included:
variable length addresses
decoupling of transport and network-layer addresses
clear separation of endpoint-id/locator (more later)
routing aggregation/abstraction

"math is hard, let's go shopping" -- solving the
real issues was set aside, people focused on
writing packet headers instead

identity -- what's in a name
think of an "endpoint-id" as the "name" of a device
or protocol stack instance that is communicating over
a network
in the real world, this is like your "name"--who you are.
a "domain name" is a human readable analogue

persistent--long term binding, stays around as long as
machine is up
ease of administrative assignment
hierarchy along organization boundry (like DNS), not
stay the same no matter where in the hierarchy you
Globally unique!
unlike human names.  ^_^

Locators: "where" you are in the network
think of "source" and "dest" addresses in routing and
 forwarding as locators
real-world analogy is street addresses or phone numbers.
typically some hierarchy (like address), or like
historical phone number (before portability!)

Desireable properties of locators:
hierarchical assignment according to topology (isomorphic)
dynamic, transparent renumbering without disruption
unique when fully specified, but may be abstracted to
reduce unwanted state
variable length
realworld--don't need to exact street address in Australia
 to fly there
Possbly applied to traffic without end-system knowledge
effectively like NAT, but doesn't beak end-to-end

Why should I care?
v4/v6 there are only "addresses" which serve as both
endpoint-ids and locators
this means they don't have the desirable properties of
assignment to organizations is painful because use as
 locator constrains it to be topological
exceptions to topology create additional global state
renumbering is hard; DHCP isn't enough, sessions
 get distrupted, source-based filtering breaks, etc.
Doesn't scale for large numbers of "provider-indep"
or multihomed sites

why should I care?
currently, v6 is only a few hundred prefixes; won't be
a problem until it really ccatches on, at which point
it's too late.
larger v6 space gives potentially more pain
NAT is effectively id/locator split--what happens if
 NAT goes away in v6?
scale of IP networks still very small compared to what
 it could grow to
re-creating the routing swamp with ipv6 with longer
 addresses could be disasterous; not clear if internet
 could be saved in that case.
Been ignored by IETF for 10+ years
concepts have been known since 60s.

Can v6 be fixed?  And what is GSE, anyhow?
Mike O'Dell proposed this in 1997 with 8+8/GSE
keep v6 packet format, implement id/locator split

basic idea: separate 16-byte addrss into 8-byte EID
and 8-byte routing goop/locator
change TCP/UDP to only care about 8-bytes.
allow routing system to muck with other 8 bytes in-flight

achieves goal of EID/locator split while keeping most of
IPv6, hopefully without requiring  new database for
EID-to-locator mapping
Allows for scalable multi-homing
Renumbering can be fast and painless/transparent to hosts.

GSE issues
problems with it--incompatible changes to TCP/UDP
in 1997, no IPv6 installed base, easy to change;
now, v6 deployed, is it too late to change?
violation of end-to-end principle.
perceived security weakness of trusting "naked" EID
(steve bellovin says this is a non-issue)
mapping of EID to EID+RG may add complexity to DNS
depending on how implemented
scalable TE not in original design; will differ from
v4 TE, may invole NAT-like RG re-wriete
currently not being pursued--killed by IETF politics.

GSE is only one approach
isn't only way to do this, but is a straightforward
retro-fit to existing protocols
other approaches:
[see slide, flipped past pretty quickly]

big problem, *no* approaches being pursued.

what about shim6/multi6
approx 3-year-old IETF effort to retro-fit an
endpoint-id/locator split...
[wow, he's running low on time, cranking through it]

Nobody really likes it, but for different cases/reasons.

What if nothing changed
how about a "thought experiment"
make assumptions about v6 and internet growth
take a guess at growth trends
pose some questions

cloudy crystal ball
v6 deployed in parallel to v4, widely adopted
v4 will be predominant; will be used indefinitely
v4 routing state growth will continue to grow at
superlinear rate, up to or beyond address exhaustion
will just get chunked up smaller and smaller
routers in DFZ zone will continue to need more and
more state.

v6 prefix assignments allow orgs to aggregate their
space into a single prefix.
shim6 won't see large adoption
v4 style multihoming will expand into v6

Questions to worry about
how much routing state growth due to orgs needing
multiple prefixes?
will growth rate decrease to number of ASes?
 in v4, prefix to AS is 6:1, will it be 1:1 in v6?
can we reduce unintentional deagg?
what is churn rate?
can we reduce intentional deagg?
if we add more state to routing for security, what
 happens to CPUs? (SBGP/soBGP)

Geoff Huston's v4 BGP growth report
starting to look like quadratic growth...

Jason's analysis: future routing state size
v6 widespread adoption *will* happen; what will it
look like?
looks at v4 routing table to decide what likely
scenario is.
Based on Tony Bates CIDR report.
prefixes: 180,219,  111K aggregated; so 1/3 are
traffic engineering prefixes
so, current 180K v4 prefixes
            21K v6 prefixes
            61K v6 TE prefixes
            50-150K internal routes for tier 1
            40-120K customer VPN/deagg'd routes
            352-532K prefixes total

a single AS that has multiple non-contiguous
assignments that would still announce same number
of prefixes.
with removal of NAT, will people who hid multiple
blocks for TE behind NATs, will they need more
what about new v6 only networks for mobile phones,

intentional deaggregates seems to be steepest
quadratic curve on his chart.

He plots UUnet internal deaggs, they are also
650K to 1M routes in 5 years
in 10 years, 1.1M low to 1.8M on high side.

Marshall Eubanks on ppml
Are these numbers ridiculous?
How many multi-homed sites could there really be?
consider as upper bound the number of small-to-med
businesses worldwide,
suggests up to 6M SMB prefixes would show up.

Jason's conclusion--make sure we know what we're
getting into when we talk about v6 without fixing
the routing hierarchy.  Can router vendors build
boxes to solve this, or do we need to fix the

Vince notes the hardware vendors loved the quadratic
growth in the late nineties, with 3 year turnovers;
helped Cisco become the giant it is.

IETF Been there  IAB/IESG is actively hostile
ITU (enemy of internet for years)
[email protected]
[email protected]
[email protected]

bunch of recommended reading slides.

10 minutes over into lightning talks already.
5 minutes of questions, and then MUST move on.

Q: Steve Bellovin, columbia university--been pushing
id/locator sep since 1986; nobody had been working
on it; Vint Cerf said he tried it way back when, it's
hard.  He does note this still has open denial of
service challenges that need to be addressed; but
Steve does agree they aren't show stoppers.

Q: Randy Bush, IIJ, been discussing this know for quite
some time--where to talk about it?  it's been talked
about everywhere, problem is the people who need to
write it are burned out.  Alain Durand is a perfect
example of ipv6 usage that doesn't pollute the global
Should we bother, or just resign ourselves to signing
up for buying new routers every 3 years?

Q: Alan--is there a mathematical model that looks at
the different routing protocols, BGP in particular,
to show where it simply will fail to converge?
A: Not sure--NSF, Craig Labovitz did some research
on it; imagine a perfect network, infinite CPU,
zero latency; can someone do the math and find out
where the hard ceiling is?
Sounds like a fine research question.

Q: Jared Mauch, NTT, what alternatives do we have for
BGP, or what are we doing to resolve that issue.
A: problem isn't BGP, it's the amount of state
being held, regardless of how you pass it around.
Q: Jared notes operator burnout is significant, and
the frustration with IETF is serious; can we solve
this on our own outside the IETF?

Q: Tony Hain, Cisco.  Security infrasctrure built around
overload; that it can look at locator to know 'who' is
being talked to.
Mike O'Dell also responsible for part of the problem
if you are going to play, there is ONE global default
free zone.  Does that assumption make sense?
Take it up on the mailing list, on architecture discuss
and ipmh.

Q: Rob Seastrom: fundamental disconnect, rough consensus
and working code disconnect.  Nobody has working code,
so this discussion is simply flailing.  Discussion
on mailing list may be bringing vendors to table.

On to the lightning talks!!