North American Network Operators Group
Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Anycast 101

  • From: Iljitsch van Beijnum
  • Date: Fri Dec 17 15:57:04 2004 
On 17-dec-04, at 19:43, Paul Vixie wrote:
i don't think iljitsch is in a position to teach an "anycast 101" class.
If anyone feels they can do better, please step up...


here's my evidence:



note-- harald asked us to move this thread off of [email protected], so i've done
that.

iljitsch added [email protected] back to the headers in his reply to me. i'm
taking it

back off again. iljitsch, please leave it off, respecting harald's
wishes.



Hey! I missed this one. I'm on dnsop but it's pretty low on my to-read
list.



Unfortunately, your evidence contains its share of errors so I'm not
sure if you should be teaching the class either.




... It's possible for bad things to happen if:





1. some DNS server is anycast (TLD servers are worse than roots
because the

root zone is so small)

2. fragmented UDP packets or TCP are used as a transport

3. a network is built such that packets entering it through router X
may

prefer a different external link towards a certain destination than
packet

entering it through router Y

4. a customer of this network is connected to two different routers

5. the customer enables per packet load balancing




#1 and #2 are normal, even though fragmented udp isn't very common
nowadays.

#3 is extremely common. #4 is normal for high-end customers. and #5
will

only affect customers whose ISP shares an IGP with the anycast -- in
other

words, "other customers of the same ISP".



Nope. Consider:



+-------+ +-------+

|ISPrtr1+---+ACinstA|

+------+---+---+---+ +-------+

|source| |

+------+---+---+---+ +-------+

|ISPrtr2+---+ACinstB|

+-------+ +-------+



Where the anycast instances exchange routing information using BGP.



If there is no special BGP configuration in effect, the ISPrtr1 will
prefer the path to anycast instance A and 2 to B, because the external
path takes precedence over a same length path that's learned over iBGP.



The current Cisco multipath BGP rules require the whole AS path to be
the same (which would be the case in this diagram if both anycast
instances use the same AS number), but older IOSes only require the
next hop AS and the path length to be the same.




Now the question is: how do we deal with this? I don't think removing

anycast wholesale makes sense and/or is feasible. Same thing for
declaring

per packet load balancing an evil practice.




as i said the other day, "all power tools can kill." if you turn on
PPLB

and it hurts, then turn it off until you can read the manual or take a
class

or talk to an expert. PPLB is a link bundling technology. if you
turn it

on in non-parallel-path situation, it will hurt you, so, "don't do
that."



Yes, per packet load balancing will cause reordering, and if that's an
issue you shouldn't use it. But if with pplb packets end up at two
different hosts, that's not the fault of the people who invented per
packet load balancing or the people who turned it on, but the fault of
the people giving the same address to two different hosts.




A better solution would be to give network operators something that
enables them to make sure load balancing doesn't happen for anycasted
destinations. A good way to do this would be having an "anycast" or
"don't load balance" community in BGP, or publication of a list of
ASes and/or prefixes that shouldn't be load balanced because the
destinations are anycast.





since PPLB won't affect BGP (since BGP is not multipath by default),
this is

not an issue.



If the uncommon network setup exists, and pplb is turned on, the
problem can manifest itself. The fact that someone had to turn on a
feature that's turned off by default is immaterial. (There is no BGP by
default to begin with.)




and they would know that PPLB is basically a link bundling
technology used

when all members of the PPLB group start and end in the same
router-pair;




It doesn't make much sense to have multiple links terminate on the
same

router on both ends as then both these routers become single points of

failure.




i don't even know what conversation we're in any more. why does it
matter

whether they are single points of failure, if this is the
configuration for

which PPLB was intended?



There is no requirement that all packets between two hosts follow the
same path. So people who pplb have the IP architecture at their side,
unlike those who implement anycast. So a little less blaming the victim
would be in order. (Well, if there are any victims, because all of this
happening is pretty unlikely.)