North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: exponential route prefix growth, was: Re: The Cidr Report

  • From: John Todd
  • Date: Fri Sep 22 17:51:08 2000


On Fri, Sep 22, 2000 at 03:44:50PM -0400, Vijay Gill wrote:
> On Fri, 22 Sep 2000, Kai Schlichting wrote:
>
> [ http://www.employees.org/~tbates/cidr.plot.html for a plot ]
>
> > of engineering experience announcing 32 /24's out of their scattered
> > POPs in addition to their /19 aggregate because they can't get their
> > load balancing right without it or have never heard of the 'NO_EXPORT'
> > attribute (I could point fingers now)?
>
> I suspect data from inside providers with aggressive filtering policies
> might prove to be interesting.
>
> Some promising local providers were past the 150k mark a while ago for sum
> (internal + global bgp) routes and were growing superlinearly the entire
> time because of the ever increasing number of customers.

I have in one location over 100k prefixes received from
one nice large international provider:

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
a.b.c.d 4 701 10776831 107250 13024503 0 0 1w4d 105250

My primary concern is that the current growth is going to
outscale the medium sized providers as the one vendor that
the above output came from doesn't appear to be providing enough memory
in their equipment, or have enough memory options available.

I am not predicting "the internet will end in 9 days, repent",
but it is some cause for concern in the next 6-12 months.

What would be nice is if someone were to put together an
auto-nagger similar to the "your dns server is lame" of the old days
messages. Back when the network tables were about 40-45k, I would read
the output of "sh ip b" and send messages to people asking them to
aggregate, but those days are over... (at least for me..)

- jared

--
Jared Mauch | pgp key available via finger from [email protected]
clue++; | http://puck.nether.net/~jared/ My statements are only mine.
END OF LINE | Manger of IP networks built within my own home

Take a look at:

http://www.fox-den.com/routes/

This is a plot of routes v. paths on a router that peers with or at least sees the tables from a few large upstreams. Please ignore the large gaps in the chart where I <ahem> did some configuration that was contrary to the correct functioning of my statistics collection. This is a crude measurement, but it's useful for trending. I'm sure someone at CAIDA has done better research than my stupid-simple MRTG graphs, but I haven't seen theirs mentioned yet so I'll throw my $.02 in first.

What is interesting in the growth difference between the two growth patterns. An interesting trend shows; there is a much larger increase in the number of paths than in the number of routes. I would assume that this indicates a growth in the interweaving of networks, or at least the interweaving of transit providing relationships. I suspect the interweaving of interconnections is growing at a similar rate, but proof of this is invisible with only a few BGP perspectives.

Perhaps the more vital piece of information in this discussion is not the sudden growth of routes, but the growth of paths. The de-aggregation of routes (though I have done no research to prove this) seems to me simply a response to redundancy/load distribution issues introduced by current route selection algorithms. I think we're identifying one of the symptoms, but not the root cause of the growth in routes. * It seems that we should be saying that there is a growth in paths, not _just_ routes, and path growth with current implementation methodologies and reasoning implies route growth. *

We come back to "BGP only works the way most people expect it to in a multi-home situation when you de-aggregate a route from one of your upstreams and announce it all the time to all your peers." Yes, there are many other reasons that one would de-aggregate and re-announce either back into the primary with the aggregate or others; however, I think that the (obvious) basic reasoning for the bulk of path/route announcements is redundancy and load sharing.

The demand for "always on" backup ("announce all the time even in directions in which you pad the path") and cost efficiency ("we're paying for that second line, so we'd better get some use out of it") lead to increase in paths that need to be visible, and thus routes that need to be visible.

Why in the last year has this been so large? You've got me there, but I'll take a swing at it. Probably something to do with the collapsing prices of bandwidth, the dispersion of BGP know-how (or the "how to set up BGP for your enterprise for dummies" books/FAQs, at least,) and the sudden expectation that all Internet traffic is mission-critical and mustmustmust always be available. The bubble of hype is finally moving down the hose and causing operational issues such as this.

It's the nature of the beast; as complexity grows... complexity grows. So how does this get solved? The complex solutions of multi-provider NAT are possible, but practically impossible in a business environment where FUD rules the day. This is really a matter best discussed after at least 6 hours of deep thought, which is about 5:55 short of what has occurred for this message. :)

JT