North American Network Operators Group
Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Phantom packet loss is being shown when using pathping in connection with asynchronous routing - although there is no real loss.

  • From: Matt Buford
  • Date: Tue Jun 06 14:36:32 2006 
"Gunther Stammwitz" <[email protected]> wrote:
I have customers who are complaining about packet loss and they are
providing me with MTRs and pathpings (that's some sort of traceroute that
pings every hop it sees several times - comes with windows xp) that show the
loss starting at my routers and ending at their server (=the last hop). All
users are coming from a (dialup-)network where the way from them to our
servers are going via a carrier different than the carrier we are using to
route the traffic back to the dial user.
The interesting thing is that there is no loss at all when the users either
use a ping instead of this pathping/mtr-stuff or when I perform a ping or
even an mtr on my server in direction of the dialup customer.

The nasty thing is that there is de facto NO LOSS on the line but the users
is seeing some sort of phantom loss.

The problem immediately disappears when I change to way back to the same
carrier as the way to us so that we have synchronous routing again.

My assumption is that pathping and mtr somehow get irritated by the icmp
messages due to a wrong timing or something like that. Any ideas?
Try varying the mtr interval, such as "-i .1" (must be root for <1). Does the packetloss significantly increase with this faster mtr? Try slower "-i 10". Does the packetloss significantly decrease or go away?

If the answer to both above questions is yes, then I would suspect ICMP rate limiting.

You could also try varying the speed of ping. Windows is pretty limited, but on unix you can do things like .1 second intervals ("-i .1" as root). Does a faster ping trigger this apparent loss? If so, ICMP rate limiting.

The only part that I don't get is that you can mtr to him without packetloss. Although the path in-between may be different, the final hop packetloss should exactly equal what he sees when mtring you. A round-trip is a round-trip, and results should be identical regardless of who originates. I can't think of any way this would be different unless echo and echo-reply were being rate limited independently.

My home ISP (apartment ethernet "t1" service, which is actually multiple T3s) has a Packeteer or something along that line. If I use ping, everything is fine since it goes so slow. If I use MTR, it works fine for the first few seconds then sees >90% packetloss on all hops from then on once the rate limiter burst bucket runs dry. Of course, TCP still sees no packetloss even when mtr is seeing this heavy rate limited loss...