North American Network Operators Group
Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Reducing Usenet Bandwidth

  • From: Eliot Lear
  • Date: Sun Feb 17 18:13:51 2002 
At 11:57 PM 2/17/2002 +0100, Iljitsch van Beijnum wrote:

On Sun, 17 Feb 2002, Eliot Lear wrote:

> This is the art of content delivery and caching.

Actually _delivery_ is only part of the problem: it assumes the content is
available, people know enough about it to be able to decide they want it,
and they know where it is and how to request it. Obviously, delivery is an
important aspect of the whole process to optimize, since it takes a lot of
bandwidth, depending on the type of content. But distributing the
meta-information is even harder, and potentially more expensive.
The only expensive part about dealing with the meta-information is adapting existing technology to point at URLs. Passing the meta-information is several orders of magnitude less expensive than the actual files themselves. Managing the meta-data (i.e., CPU/memory) is negligable compared to the cost and latency of retrieving that data. The retrieval of XOVER information and such scales well. That's been the lesson of the last few years. And content delivery networking mechanisms can be used for it.

The
failure of Usenet to effectively do it demonstrates this: because
selection is pretty much impossible, you have to deliver everything to a
place very near the potential user, even the stuff that is of no interest
to any user.
To call USENET a failure is a bit of a stretch. But its scaling point is on the side of the reader. That is solved today through stronger search capabilities of various search engines that were merely a gleam in several people's eye, even as late as 1990. It wasn't until WAIS came about that USENET became more searchable, and then the economics began to shift. Not that people didn't try to do selection. Brad Templeton had a semi-automated feedback mechanism with Clarinet that he gave away to his customers.

In reality, people don't want to think about it. How much am I willing to
pay for the privilige of posting this message to NANOG? And you to read
it? If we both apply the hourly rate we bill our customers to the time we
spend on it (because we could have been doing work that actually pays
money instead), probably more than we realize. On the other hand, If I had
to cough up some money right here, right now to post this, I probably
wouldn't.
Nobody wants to think about it until real money is involved. And while text discussions don't involve a whole lot of real money, this conversation was started by people who are tired of transporting warez and having to pay for it.

Eliot