North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Slashdot: Providers Ignoring DNS TTL?

  • From: Dean Anderson
  • Date: Fri Apr 29 23:58:17 2005

On Sun, 24 Apr 2005, Steve Gibbard wrote:

> 
> On Sun, 24 Apr 2005, Robert M. Enger wrote:
> 
> > Steinar:
> >
> > There is a large body of work from competent and well known researchers 
> > that assert the claim.  I certainly lack standing to question their 
> > results.
> >
> > Empirically, download speeds to home are nearly cut in half (18Mbps) 
> > from sources that are subjected to packet reordering along the path.
> 
> I'm trying to sort out the various claims here, since I think right now 
> this is a case of people talking past each other, and arguing completely 
> different points.
> 
> First of all, let's ditch the term "PPLB."  The usual alternative to per 
> packet load balancing (what's been being talked about here) is per prefix 
> load balancing, which would also be "PPLB."  The abbreviation is therefore 
> more confusing than anything else.

Err. No, that would be worse. "Per prefix" load balancing is an artifact
of the Cisco route cache. The route engine (ie the route table) isn't
queried for every packet. Instead the route in the route cache is used.  
One doesn't configure "per prefix" load balancing. One configures load
balancing, which adds multiple routes into the route table.  The route
cache then causes only one of these routes to be used.  On cisco, to
enable PPLB, you turn off the route cache.  On Juniper, you configure it
to put multiple routes in the route table.  Its actaully more likely to
happen on Junipers, because unless you configure additonal policies, you
get load balancing on divergent links as well as non-divergent links.  On
Cisco, the route cache is controlled on a per-interface basis.

> Now, onto the argument that's going on here.
> 
> Dean says "per packet load balancing is coming," and then goes on to 
> assume it's going to be used in such a way that it will cause packets to 
> route through widely divergent paths.

Almost a correct statement of my position. But if its used in any
case--anywhere--even on internal links, in any condition other than
between exactly two routers, it will cause a problem with Vixie's TCP
anycast.

We aren't seeing a lot of problems yet because few people are using PPLB,
and fewer are using TCP DNS, and so few are using Vixie's TCP anycast.

If we change any our terms in this discussion, it should be to change our
use of "TCP anycast".  The behavior described by Vixie for DNS TCP
"anycast" is unapproved** and non-standard. It does not conform to RFC
1546 TCP Anycast. RFC 1546 outlines changes to the TCP stack to support
TCP anycast.  To avoid confusion, I propose we call it "vixie-cast".  And
I also want to emphasize that RFC 1546 TCP anycast doesn't have any
problems with PPLB. And of course UDP anycast also has no problem. It is
only "Vixie-cast TCP" that has a problem.

** One of the criticisms of DNS Root Anycast is that Vixie recommended
deployment of anycast to root server operators without first discussing
this on the DNSOP WG.  Dr. Bernstein first brought it to the WG's
attention in 2002. And one of the recent discoveries is that a good number
of DNS root server operators are using it, and had assumed the Vixie was
the IETF liason communicating with the IETF. The IETF has never approved
of this, and in fact, Vixie's version of TCP anycast is not what is
described in RFC 1546. Vixie's version really shouldn't be called anycast,
since it creates confusion with RFC 1546 recommendations.  Indeed, if RFC
1546 recommendations for TCP anycast were widely implemented in TCP
stacks, then there wouldn't be any problem. The problem is caused by
violating RFC 1546.  

> So, as far as I can tell, everybody except perhaps Dean agrees that:
> 
> - Used incorrectly (on divergent paths), per packet load balancing can
>    cause packet reordering.

I agree that used on divergent paths, a problem occurs with Vixie-cast.  
Additionally, packet reordering can happen, which can cause problems with
TCP stacks that aren't able to handle insertions. This reordering problem
is symptomatic of a badly implement TCP stack. Other things can cause 
reordering, besides PPLB.

> - Used correctly (on non-diverging paths), packet reordering doesn't
>    happen often.
>
> - Packet reordering is bad, and should be avoided.
> 
> I'm less clear on Dean's position, but I think it's something along the 
> lines of:
> 
> "Per packet load balancing over divergent paths is coming, by fiat from 
> marketing departments even if engineers don't like it, and anything that 
> doesn't play well with it needs fixing."  While Dean focuses on anycast, 
> that would presumably extend to TCP and to anything jitter sensitive, such 
> as streaming audio or video.

No, that isn't quite it. (While it may indeed come by fiat of marketing,
but I don't know that and don't assert that) But I assert that only
"vixie-cast" has a problem.  Streaming video and audio is thought to be
reorder sensitive, but this is the same problem as with TCP reordering,
and has a similar solution. If the jitter buffer can do inserts
efficiently there is no problem.  And, of course, the out-of-order packets
must arrive before they are to be played.  Some jitter buffers can do this
inserting well, others can't.

Indeed, TCP stacks that properly implement RFC 1546 TCP anycast
recommendations will work without problem.  The problem is that
"vixie-cast" doesn't conform to RFC 1546.

Also, the reorder problem only causes a performance problem and never
causes incorrect operation. Vixie-cast causes incorrect operation.

		--Dean

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000