North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Traceroute versus other performance measurement

  • From: smd
  • Date: Wed Nov 29 15:45:14 2000

Mmm, network analysis stuff on NANOG.  Scary!

| Do I understand it correctly that the "invariant 5 tuple" refers to
| 5 non-variable aspects of this performance test, being namely:

Tuple definition, from the CS Jargon file, a copy of which is here:

Note that a 5-tuple is basically a set of 5 elements/objects/records.

The "invariant 5-tuple" here just has to do with 5 things which don't
change in the test.   There is another element which is unchanged
during a ttcp run: TOS/diffserve, thus there is really a (or an at-least)
6-tuple invariance.

In practice, only 5 of the elements in that n-tuple are used in
making a forwarding decision; the choice of the fifth could well
be vendor/release-dependent, at least judging by what people are saying here.

| E.g.: performance measurements will gravitate towards a result not
| representative for the totality of all traffic, as the 5 variables
| are rather fixed, or at the least fixed for the duration of the test ?

Well, if you are making a big flow, and the elements used by routers
to make a hash which is used to select which of several equal-cost paths
to use don't change, then the routers will use the same single path 
throughout the ttcp test.  (Even if the hash->path mapping is adjusted,
the ttcp test packets hash the same, so path flutter doesn't really
change the principle).

Thus if you have six 10Mbps lines in parallel, the ttcp packets
will hash the same way, and use a single 10Mbps link (at a time).
This is good, because TCP does not like out-of-order packets, and
because traffic loads are fractal, out-of-order packets are likely
where there are multiple paths, and if there are many multiple-paths
this can lead to the TCP sender losing its brains with duplicate ACKs
pushing the congestion window right through the floor.

| Has intelligent switching and forwarding (Layer-4 and up) started to
| make such a significant impact on end-to-end performance that all
| traditional performance tests must necessarily be flawed ?

Research question.   My guess: probably not.

A better question, maybe the one you want to ask, is how well
does the Internet deal with lots of equal-cost paths?

The answer: small numbers are OK, large numbers are probably not OK,
and moreover, the goal of avoiding out-of-order TCP packets constrains
any given TCP flow to one path, which means that that TCP flow will not
be able to run any faster than the slowest component, across any given
link.   If that component is a bottleneck, oops.

| (there must be a reason why my MRTG graphs of HTTP GET's to websites
|  are so much more consistent than icmp pings...)

Consistent?  I'm not following.

Maybe there is a large-numbers issue here: if you are plotting lots
and lots and lots and lots of flows, and only looking at a few pings,
then if there is frequent very-short-term queueing delays at work,
you will see them more clearly with the individual pings.   A large
number of pings aggregated together will probably converge upon whatever
you see when you aggregate a large number of HTTP GETs.

cf.  and the cool toy at

Note that the tendency of small numbers of short-term samples to
be wildly different than large-number samples when looking at ping
(which ultimately is a measurement of queueing behaviour) is an
important property of the Internet.   The short-term behaviour is
_fractal_, and is fractal across a surprising number of _generations_
(where a generation is, for example, 0.001 sec to 0.01 sec, that is
an order of magnitude increase base ten).

This really really messes up the fine-tuning of problem identification,
because again the law of large numbers suggests that as the Internet
grows in terms of numbers of users, more of them will see shockingly
bad behaviour if they ping somewhere for a few seconds.   :-(

(In other words, Danie Senie was right with respect to users seeing
strange things and asking about them).