North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Slashdot: Providers Ignoring DNS TTL?

  • From: Richard A Steenbergen
  • Date: Sun May 01 04:23:48 2005

On Sun, May 01, 2005 at 12:44:09AM -0400, Dean Anderson wrote:
> 
> > Modern Cisco routers do not use a "route cache", 
> 
> You'll need to define what you mean by "modern" with respect to cisco.  
> This statement seems to be incorrect.

For someone with so little clue, you sure do manage to talk a lot. :P I'm 
not under any delusion that anything I say here will help you understand 
anything, but if it helps some other soul than it may be worth it.

Historically, routers were entirely CPU driven beasts. The most basic and 
fully functional type of routing lookup (and also the slowest) was known 
as process switching, where the destination address was fully evaluated 
against the "master routing table" (RIB) on the CPU. The data structure 
used to hold this RIB, one of the easiest algorithms used to implement 
"longest prefix match" lookups, is known as a PATRICIA (Practical 
Algorithm To Retrieve Information Coded in Alphanumeric) tree.

Unfortunately, these full RIB lookups were (and still are) a relatively 
slow process. In Cisco speak, a "route cache" is any of a variety of 
mechanisms that are used to cache routing lookups, using a mechanism which 
is "faster" than a full RIB lookup. Some of these mechanisms include the 
"fast cache" (where the most popular destinations are stored in a smaller 
cache after the first packet is looked up via process switching), "flow 
cache" (where individual layer 4 flows are stored), and "cef" or Cisco 
Express Forwarding.

On modern Cisco routers, CEF is the only thing you will find used to 
actually do routing lookups. However, CEF is more of a brand name than a 
detailed description of how the routing lookup occurs, and the 
implementation varies greatly from platform to platform. The only thing 
that CEF really describes across all varieties (other than the fact that 
you will soon be experiencing its other meaning, the Customer Enragement 
Feature :P) is that the route cache will use a "pre-populated" FIB 
(forwarding information base).

A FIB is a data structure which exists solely to do routing lookups, and 
is created from a normal RIB. A classic software implementation of a FIB 
uses a multi-bit trie (mtrie) to fully map the destination next-hops of 
the entire address space. This consumes a little bit of memory (several 
megabytes), but gives you a data structure which is fully "pre-populated" 
and delivers consistant results. Tihs means that there is no mode where 
the first lookup can take longer than the rest, the memory usage does not 
increase as you look up more destinations, and the number of memory 
accesses is roughly the same for all addresses (vs a patricia tree where 
the number of access can vary greatly depending on the tree depth).

All of this is historical of course, as "modern" routers do all of their 
packet lookups in hardware using designs which look nothing like any of 
this. While Cisco does call everything "ip route-cache", in modern routers 
the commands are just there for historical compatability.

> > they use a fully populated forwarding table. And load balancing is
> > automatic if you have several equal cost routes.
> 
> This sounds very much like the Juniper description for the Internet 
> Processor ASIC behavior. I'd say that's worse.

Load balancing has nothing to do with CEF or a FIB persay, the FIB is just 
a good spot to slap multiple next-hops. Juniper's implementation uses a 
"pre-populated FIB", the same as everyone else, they just do it using a 
tree primitive on an ASIC and controlled by a CPU based switch board.

> > > The route cache then causes only one of these routes to be used.  On
> > > cisco, to enable PPLB, you turn off the route cache.
> > 
> > Many modern Cisco routers can perform per-packet load balancing without
> > doing process switching (but this needs to be explicitly configured).
> 
> Well, 7500 and 7200 have interface processors that can route packets using
> the route cache without interrupting the main processor. So, if you don't
> consider 7500's and 7200s to be "modern", this feature above doesn't seem
> like a big deal: They could do that before. It was called CEF and DCEF.

7200 most certainly does not have interface processors. 7500 does have 
processors on the VIPs that do forwarding lookups in a distributed 
fashion, but the same procedure for software forwarding apply, there just 
happen to be a few more CPUs floating around. DCEF is just CEF plus 
copying the FIB structure to the VIPs. And no I don't think any sane 
person would consider 7500s or 7200s to be "modern", even though you can 
still make use of them.

> > > On Juniper, you configure it to put multiple routes in the route
> > > table.  Its actaully more likely to happen on Junipers, because unless
> > > you configure additonal policies, you get load balancing on divergent
> > > links as well as non-divergent links.  On
> > 
> > Modern Juniper routers cannot do per-packet load balancing *at all*. It
> > is correct that the configuration statement says "per-packet", however
> > it is really per-flow (and this is well documented). See for instance
> > the description of Internet Processor II ASIC load balancing at
> > 
> > http://www.juniper.net/techpubs/software/junos/junos70/swconfig70-policy/html/policy-actions-config11.html#1020787
> 
> I don't have Junipers, so I'm just going by what the manual says. And your 
> link says: 
...
> I would gues that since both processers are described, that they are both 
> still supported, and that probably means that both are widely used.

The original poster is entirely correct. The original Internet Processor 
is still supported, but it is about as far from "widely used" as you can 
get. If you're still using it you have bigger problems. The IP2 is only 
capable of doing per-flow load balancing, which is probably a good thing.

> But, I should qualify that doing PPLB on diverse paths is more likely to
> happen on the Internet Processor ASIC. It would seem like the Internet
> Processor II ASIC has an architecture more like the cisco 7500s, and only
> allows per flow load balancing.

The IP vs IP2 has no architecture that is or isn't more like the 7500s. 
The IP2 is just a newer version of the ASIC on the switch fabric cards. 
Classic Juniper architecture (everything pre M320/T series) actually uses 
all centralized routing (and other forwarding/filtering) lookups.

> > I'm afraid your statements show a certain lack of knowledge about modern
> > router architectures.
> 
> I'm afraid your statements show a certain lack of knowledge about whats
> being used in datacenters to route packets. And perhaps some arrogance
> about whats "modern".  I'd still call cisco 7500 and 7200 series routers
> "modern", and they have route caches.  I don't know that much about GSRs,
> but they didn't seem to get much traction. I can't say whether they have a
> route cache or not.  And the multi-rack monster router that cisco just
> announced a while back doesn't seem to be too popular either. I don't know
> its architecture, either.  As I look around datacenters, the 7500 and
> 7200's and to some lesser extent Junipers are the workhorses doing most of
> the routing.  And that basic technology will stay around in the enterprise
> for many more years to come.

What ever happened to the days when people who didn't understand something 
would sit down, shut up, and listen to the people who did? You my friend 
don't even have a basic grasp on the old hardware (which is most assuredly 
not modern), let alone a firm grasp, let alone any idea how modern (you 
know, that stuff made in the last 5+ years) works. Instead of spouting off 
about things which you don't understand, you might do well to listen and 
learn. Well, that is if you actually want to learn of course. Maybe you 
just want to talk. :)

> And "vixie-cast" violates this rule. It imposes the new rule that "an
> internetwork must deliver to successive packets sent to the same anycast
> address to the same host."  And no one has thought much about the
> implications of that rule.
> 
> Assurances that no one can do PPLB on diverse paths offer no defense to
> having violated the design principles given for anycast in RFC 1546.
> 
> It is also objectionable to calling something "TCP anycast" that isn't TCP
> anycast according to RFC 1546.

I've successfully managed to tune out this thread until now, so I don't 
know what babble I've missed so far, but let me sum it up for you in 
no-nonsense terms:

Nothing says that you can't have out of order packets on the Internet 
until you are blue in the face. However, it tends to do very nasty 
things to the TCP algorithm, which makes it perform poorly. If you are 
fine with poorly performing TCP then you go right ahead and re-order your 
packets, but I know that I'm not fine with this, nor are my customers or 
the vast majority of other Internet users. Therefore, if vendors want to 
design a product that end-run the problem by maintaining packet ordering 
when they load balance, good for them.

-- 
Richard A Steenbergen <[email protected]>       http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)