North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Keynote/Boardwatch Internet Backbone Index A better test!!!

  • From: John Leong
  • Date: Fri Jun 27 18:34:24 1997

> for an exmple of somewhat more complete and better designed
> benchmarks of this type:
> 
> http://www.inversenet.com/about/backgrounder.html#2

We make web measurements from over 800 points (POPs) on the Internet for
roughly 20 ISP's using a standard basket of 10 popular web pages (porn
pages excluded given there is the believe it will drive down the
productivity of our operation staff :)  

We control the test by having every ISP runs through the same set of
pages at roughly the same time.  (By the way, to closely emulate
consumer end user's experience, we specifically use a [painful] test
harness to drive Netscape 3.0 on Windows95 over boring 33.6 modem line
... would have love to use a UNIX script with our own custom browser
over our direct link into the net instead ... but then it is not what
consumer users are doing ... oh well).

Even with the extensive matrix of measurements, the only claim we dare
to make is that it approximate a end user experience index a la Dow
Jones's basket of companies.  (By the way, with few interesting
exceptions, mostly limited by the size of the final pipe - the modem
link).

We think there are way too much variables involved to allow us to
defintively partition out the components such as the page structure (how
many other embedded objects we have to retrieve, compressibility by
modem of those objects etc.), the Web server load at the time the
measurement was made, the load of the web server's LAN, the load and
size of that LAN's pipe into the server's ISP, that ISP's network
performance, its peering performance with the end user's ISP (and
transit links if any), the user's ISP's network performance  .... etc.
etc. etc.  Even with extensive analysis, it will be difficult for us to
derive the backbone performance component from our end to end
measurement.

If it is the underlying network performance one wants I think a
different measurment infrastructure such as those being developed by
Paxon/Mathis/Jamshid (NIMI), Almes/Barber (Surveryor), and Graig
Labovitz (Merit) ... using tools such as Jacobson's Pathchar will be
much more appropriate.  

Regards,
John Leong