North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Cogent/Level 3 depeering

  • From: Richard A Steenbergen
  • Date: Wed Oct 05 21:42:38 2005

On Wed, Oct 05, 2005 at 04:51:12PM -0700, JC Dill wrote:
> Depeering never makes sense to me.  Customers of both companies are 
> expecting their vendor to connect them to the customers of the other 
> company.  These customers are each paying their respective vendor for 
> this service.  Why should one vendor pay the other for this traffic that 
> is mutually beneficial to them while the other pays nothing?  Why does 
> the amount of bandwidth or the direction it travels make any difference? 
> The customers are PAYING for the bandwidth.

Point being, they don't both feel that the traffic is mutually beneficial. 
Peering relationships end when either one of the parties decided they 
don't want to swap bits for free any more, at which point notice is 
given, and the two parties make arrangements to deliver their traffic via 
alternate means.

> If each vendor pays their own costs to a peering point then they should 
> be passing that cost onto > their respective customers as part of the 
> *customer's* bandwidth bill.
> "Even though you pay us for internet access, we want this other network 
> to pay us TOO so that we get paid double for transmitting your traffic 
> to and from that other network rather than doing a mutual exchange like 
> we do with other big networks."

At the risk of wasting everyone's time by explaining how and why peering 
and transit works on a mailing list which is supposed to be loaded with 
people who are experts on the subject (HAH), I think you're missing quite 
a few concepts that are central to the debate.

First of all, do you have any idea how much it costs to operate a sizable 
network? Empty routers alone cost hundreds of thousands of dollars, then 
each OC192 or 10GE card costs hundreds of thousands of dollars more. You 
need routers in pairs for reliability, interconnected, and you need an 
aggregation layer to efficiently terminate customers. You need to 
interconnect individual nodes across a diverse backbone, if you're 
operating a fiber network you need to pay for LH DWDM equipment which 
costs hundreds of thousands of dollars for every card used along the path, 
which must be repeated many times, equaling hundreds of cards. You then 
need to operate metro fiber networks to deliver the bits to the specific 
buildings where the customer is located, which means more fiber costs, WDM 
costs, possibly telco circuit costs. We're already in the hundreds of 
millions of dollars on the cheap side, billions on the expensive side, of 
capex, plus millions in operational expenses. And we haven't even covered 
colocation, powered, techs, crossconnects, peering interfaces, traffic 
management, network engineers, corporate overhead, sales staff, etc.

And Cogent sells transit for $10/Mbps. How much traffic do you think is 
OUT there to pay for this?

Yes they lose money hand over fist doing it (though not nearly as high a 
burn rate in absolute dollars as some other networks *cough*), and it can 
only be sustained by investors pumping money into the system over and over 
again, under the delusion that somehow they'll come out of it making 
money, but it does happen.

Despite the delicious irony that Level 3 was the original disruptive 
pricing carrier many years ago, Cogent is the disruptive pricing carrier 
now. They actually run a pretty lean and efficient shop, which is how 
they're able to keep the losses as low as they do, but Cogent makes what 
little money it does make on being able to deliver massive amounts of 
bandwidth to carriers all over the Internet for as cheap as possible. Much 
of its traffic is hosting based and outbound only, which means that they 
closest-exit ("hot potato") bits onto other networks, and leave those 
networks responsible for an extremely disproportionate amount of the 
longhaul, costing millions of dollars alone on an ongoing basis. All the 
while, Cogent undercuts the market of every other carrier who isn't as 
efficient as they are, leading to massive losses, bankruptcy filing after 
bankruptcy filing, out of court reorganizations and purchases for foreign 
companies, etc.

In case you missed it, this industry is in a state of utter financial 
disaster on almost all fronts right now. Billion dollar companies are 
surviving on scraps and what they can sell from previously purchased 
infrastructure at any price, with no ability to handle the need for 
upgrades when the existing infrastructure runs out. Networks everywhere 
are running out of money, pissed off, and realizing that the access to 
their customers which they provide for free to Cogent is what is enabling 
Cogent to operate and offer the pricing that it does. They don't feel that 
it is of mutual benefit any more, and they want it to stop. Level 3 also 
delivers much more "expensive" traffic, is a real Tier 1, has almost 6x 
the customer base of Cogent by prefix count (though probably roughly the 
same or even less overall traffic). It is not unreasonable to expect them 
to want to capture as much settlement as possible from networks putting 
traffic into theirs, infact this is a critical requirement in order for 
them to compete. As with any Tier 1 network, a huge amount of the traffic 
stays on-net.

Of course reality is far more complex than even that little generic 
description above. Peering is highly political, affecting sales, potential 
customers, and industry goodwill positively and negatively whether you do 
or don't peer, and whether you do so with the same policy ("fairly") for 
all networks. If you look at Cogent's previous routing, it is blatantly 
clear that they were never going for ratio balance with Level 3. It is 
entirely possible that Level 3 was asking them for traffic with which they 
could bill their customers, rather than delivering that traffic via the 
competition. Of course the only thing you're going to see on this list 
about why L3 felt the need to depeer Cogent in this particular instance is 
pure speculation, but there ARE very real and very steep costs associated 
with peering and peering policies at all levels. These costs do need to be 
taken into account, and L3 certainly needs all the money it can get its 
hand on (which is why it has depeered many more networks than just Cogent 
recently). If anything, I'm personally surprised they didn't do this 

Cogent is one of the few networks out there with the balls to put their 
customer base on the line in this way to fight such a depeering (possibly 
because their customer base consists so largely of multihomed and/or 
extremely cheap networks who are willing to endure this and other issues 
to save a buck). They certainly don't want to look weak on this issue, or 
other networks with whom they may have less than perfect peering 
relationships will certainly move on taking similar steps. What happens 
here could set the tone for many more future depeerings, or lack there of, 
and in some ways the survival of both companies and the course of the IP 
transit industry at large. Both networks are highly motivated and have 
perfectly reasonable positions which make sense to them, hence they are at 
an impass which can only be resolved through suffering. To force either 
one to peer by law or regulation would be even more disruptive to the 
industry as a whole, which may even lead to a complete collapse of the 
existing peering system as we know it. After all, if anyone can become or 
retain an "undeserved" peer by the BGP equivalent of saying "you can't 
make me route packets to you any other way", we could see this kind of 
thing happening a lot more often, trading short term customer satisfaction 
for long term economical gain. If forced to keep "undeserved" peers in the 
long term, peering may vanish completely, to be replaced by some other 
means of settlement (at least without government funded networks at the 
cost of billions).

Some of the few insights on this list on the subject of depeering have 
been provided by folks like Vijay Gill and Dan Golding, if you want to get 
a better feel for why these large networks are motivated to depeer others 
I highly recommend reading their posts before jumping to any conclusions 
that the way you see peering as a small network operator has ANY 
relationship to the way a large network operator has to see it.

Richard A Steenbergen <[email protected]>
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)