North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Network end users to pull down 2 gigabytes a day, continuously?

  • From: Stephen Sprunk
  • Date: Sat Jan 20 22:42:10 2007


Thus spake "Dave Israel" <[email protected]>
The past solution to repetitive requests for the same content has been
caching, either reactive (webcaching) or proactive (Akamaizing.)  I
think it is the latter we will see; service providers will push
reasonably cheap servers close to the edge where they aren't too
oversubscribed, and stuff their content there.  A cluster of servers
with terabytes of disk at a regional POP will cost a lot less than
upgrading the upstream links.  And even if the SPs do not want to
invest in developing this product platform for themselves, the price
will likely be paid by the content providers who need performance to
keep subscribers.

Caching per se doesn't apply to P2P networks, since they already do that as part of their normal operation. The key is getting users to contact peers who are topologically closer, limiting the bits * distance product. It's ridiculous that I often get better transfer rates with peers in Europe than with ones a few miles away. The key to making things more efficient is not to limit the bandwidth to/from the customer premise, but limit it leaving the POP and between ISPs. If I can transfer at 100kB/s from my neighbors but only 10kB/s from another continent, my opportunistic client will naturally do what my ISP wants as a side effect.


The second step, after you've relocated the rate limiting points, is for ISPs to add their own peers in each POP. Edge devices would passively detect when more than N customers have accessed the same torrent, and they'd signal the ISP's peer to add them to its list. That peer would then download the content, and those N customers would get it from the ISP's peer. Creative use of rate limits and acess control could make it even more efficient, but they're not strictly necessary.

The third step is for content producers to directly add their torrents to the ISP peers before releasing the torrent directly to the public. This gets "official" content pre-positioned for efficient distribution, making it perform better (from a user's perspective) than pirated content.

The two great things about this are (a) it doesn't require _any_ changes to existing clients or protocols since it exploits existing behavior, and (b) it doesn't need to cover 100% of the content or be 100% reliable, since if a local peer isn't found with the torrent, the clients will fall back to their existing behavior (albeit with lower performance).

One thing that _does_ potentially break existing clients is forcing all of the tracker (including DHT) requests through an ISP server. The ISP could then collect torrent popularity data in one place, but more importantly it could (a) forward the request upstream, replacing the IP with its own peer, and (b) only inform clients of other peers (including the ISP one) using the same intercept point. This looks a lot more like a traditional transparent cache, with the attendant reliability and capacity concerns, but I wouldn't be surprised if this were the first mechanism to make it to market.

I think the biggest stumbling block isn't technical.  It is a question
of getting enough content to attract viewers, or alternately, getting
enough viewers to attract content.  Plus, you're going to a format
where the ability to fast-forward commercials is a fact, not a risk,
and you'll have to find a way to get advertisers' products in front of
the viewer to move past pay-per-view.  It's all economics and politics
now.

I think BitTorrent Inc's recent move is the wave of the short-term future: distribute files freely (and at low cost) via P2P, but DRM-protect the files so that people have to acquire a license to open the files. I can see a variety of subscription models that could pay for content effectively under that scheme.


However, it's going to be competing with a deeply-entrenched pirate culture, so the key will be attractive new users who aren't technical enough to use the existing tools via an easy-to-use interface. Not surprisingly, the same folks are working on deals to integrate BT (the protocol) into STBs, routers, etc. so that users won't even know what's going on beneath the surface -- they'll just see a TiVo-like interface and pay a monthly fee like with cable.

S

Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking