Level 3 outbid Akamai on Netflix by reselling stolen bandwidth(digitalsociety.org)
digitalsociety.org
Level 3 outbid Akamai on Netflix by reselling stolen bandwidth
http://www.digitalsociety.org/2010/11/level-3-outbid-akamai-on-netflix-by-reselling-stolen-bandwidth/
61 comments
Rarely is there a clear rationale for who a middleman should extract their fee from. In practice, it comes down to who is willing to pay to be part of the transaction.
Comcast customers pay for internet access, but I doubt that agreement specifies anything about network topology. If L3 wants a special connection, and it's worth more to them than it is to Comcast, then I don't see what's wrong, in principle, with charging for it.
Regardless, the net neutrality thing certainly seems bogus. On the other hand, if L3 didn't go on the offensive with PR, they probably would have taken a larger share of the blame from users than they deserve.
Comcast customers pay for internet access, but I doubt that agreement specifies anything about network topology. If L3 wants a special connection, and it's worth more to them than it is to Comcast, then I don't see what's wrong, in principle, with charging for it.
Regardless, the net neutrality thing certainly seems bogus. On the other hand, if L3 didn't go on the offensive with PR, they probably would have taken a larger share of the blame from users than they deserve.
? Just because an agreement doesn't make 100% sense to you doesn't mean it can be unilaterally broken!
Above, I'm not speaking of any particular agreement. For the current Level3-Comcast issue, as I read that "article" they didn't break it, but Level 3 would cause operating conditions that made the existing Comcast contract no longer applicable, so they needed a new agreement of some sort.
It's good that you qualified what you're saying, because the natural assumption is that you are talking about the particular agreement under discussion.
Yes, especially when I used the companies as an example.
HN: not just news, but a lesson in communication.
HN: not just news, but a lesson in communication.
I have an internet connection from Comcast. If I only ever connect to other Comcast customers, I should get my connection for free?
I'm not sure that's at all the argument jws was making. From the customer's point of view, why should they be able to download 100 gigabytes (through Comcast's infrastructure) from an endpoint also within Comcast's network with no problem, but now they can't get any data at all from a level3 endpoint? They're taking up 100 gigs of bandwidth on Comcast's network regardless. Why does level3 have to pay extra to send that customer the bits they've requested?
Why? Because Comcast can make them. Comcast invested in infrastructure which brought subscribers to Netflix. When these subscribers have no alternative (which is apparently the case in many parts of the US) Comcast gets to do all the usual monopoly tricks. "How much is this subscriber worth to you?" Enough to pay, clearly.
For comparison, there was a little discussion recently in the UK re: net neutrality. It is unlikely to go anywhere because of the ease of switching ISPs, and the fact that the BBC - probably the biggest bandwidth sucker nationwide - is 'highly unlikely' to pay ISPs for access. If the biggest player won't pay, who will?
Business is about extracting value, not recouping costs. Amoral, not benevolent.
For comparison, there was a little discussion recently in the UK re: net neutrality. It is unlikely to go anywhere because of the ease of switching ISPs, and the fact that the BBC - probably the biggest bandwidth sucker nationwide - is 'highly unlikely' to pay ISPs for access. If the biggest player won't pay, who will?
Business is about extracting value, not recouping costs. Amoral, not benevolent.
Well yes, of course a business is going to use any tricks, dirty or otherwise, to get as much money as possible. The fact that Comcast is charging their subscribers for access to "the internet" means they ought to be obligated to take bits from anyone who will provide them. If level3 is saying "A person on your network wants these bits, here they are!", Comcast has an obligation to their customer to deliver them.
Look at both ends of the transmission: If both endpoints are inside Comcast's network, Comcast receives payments from both the sender and receiver to cover the costs. If one end point is in Level3's network and Level3 refuses to pay, only one side is paying for the connection.
Or consider just the part of the transmission that occurs within Comcast's network. Case 1: data is sent from Comcast-user-A to Comcast-user-B, both pay a share of the costs. Case 2: data is sent from Comcast-user-A to Level3-Border-Router, only Comcast-user-A pays.
Or consider just the part of the transmission that occurs within Comcast's network. Case 1: data is sent from Comcast-user-A to Comcast-user-B, both pay a share of the costs. Case 2: data is sent from Comcast-user-A to Level3-Border-Router, only Comcast-user-A pays.
This is a silly argument.
Comcast is charging its customers for access to a nebulous (but commonly-understood) entity called "The Internet". If their pricing model doesn't anticipate that users would want to access bits that exist outside of Comcast's own network (and therefore originate from someone who doesn't pay Comcast), well, it's a wonder they're still in business!
Comcast is charging its customers for access to a nebulous (but commonly-understood) entity called "The Internet". If their pricing model doesn't anticipate that users would want to access bits that exist outside of Comcast's own network (and therefore originate from someone who doesn't pay Comcast), well, it's a wonder they're still in business!
I still find the fact that level 3 decided to pay comcast the most disturbing. I do not understand the long-term financial benefit of that decision in any way.
Normally, a retailer pays a wholesaler or producer money to get goods, then re-sells them to end users.
Produces set the price, and retail outlets, with a markup, come up with a business strategy ot move that inventory to their customers.
Now - look at Wal-mart. They're big enough that they can go back to the producers and say "Hey, by the way, you're going to knock 1% off the price this year or we're going to drop you - and we can say that because YOU need US."
Same deal here.
Produces set the price, and retail outlets, with a markup, come up with a business strategy ot move that inventory to their customers.
Now - look at Wal-mart. They're big enough that they can go back to the producers and say "Hey, by the way, you're going to knock 1% off the price this year or we're going to drop you - and we can say that because YOU need US."
Same deal here.
They can still send packets to Comcast through a transit provider without paying Comcast. What they can't do is start sending 5 times as much data through their free peering agreement, and the renegotiation involves Level(3) paying Comcast. Obviously they will be paying Comcast less money than they would pay a 3rd party transit provider.
A third party who Comcast would charge to accept their traffic. Let's be honst here; wherever Netflix goes, Comcast is going to chase after them demanding money to handle their traffic.
It does sound a bit like Kroger charging Kraft to carry their cheese in the stores.. although there have been arrangements like that to improve product placement on shelves (I think).
These companies are swapping services. They both benefit from that. If one company thinks they have the power to do so, they can also extract additional payment. It's just a matter of the details on who has that power. Level3 can choose to pay transit providers (which Comcast can underbid), or they can cut off the Netflix streams (which likely pisses off Netflix quite a bit, and certainly cuts their own revenue). I suspect Comcast wins here unless the PR kerfuffle gambit works.
These companies are swapping services. They both benefit from that. If one company thinks they have the power to do so, they can also extract additional payment. It's just a matter of the details on who has that power. Level3 can choose to pay transit providers (which Comcast can underbid), or they can cut off the Netflix streams (which likely pisses off Netflix quite a bit, and certainly cuts their own revenue). I suspect Comcast wins here unless the PR kerfuffle gambit works.
Nothing about internet routing, including the cost structure, has anything to do with any sort of practicality. It's all politics and bureaucracy. Comcast and Level 3 are basically in the middle of a bureaucratic skirmish; there's nothing more sinister or unusual going on here.
The article does a decent job of introducing the concepts of peering and transit. However, it's pretty long winded and goes through a number of tortured justifications. Here's a much simpler explanation:
Comcast is charging Level 3 because they can.
Comcast is charging Level 3 because they can.
Sure, but what I am trying to figure out, as a relative layperson on this topic, is this: Is either party making some outrageous and unprecedented demand that will bring about the end of the internet as we know it? Or is this just business as usual?
It's not business as usual, which is why it's a big deal. Comcast is trying to do something new. They used to get a cut of revenue from Netflix traffic because it was on Akamai, who pay to sit in Comcast's datacenters. Now that traffic is going to come from a peer. It's the same traffic, but Comcast isn't getting the revenue, and they're pissed.
That may be the case, but if the peer made a contractual agreement that limited the peering to symmetrical traffic, then Comcast should be free to renegotiate or straight up sever the peering under the terms of any contractual agreements....if this is what is allowed by what's on the contract. While people can hem and haw about the moral and ethical implications, it seems like this is something that is likely spelled out on paper somewhere. It seems like one or both organizations is fanning the flames in the media to rile up end users.
There's no technical evidence (you can check, quite easily, who's peering with who and if there's a cost associated) that L3 and Comcast are using a purely-symmetric SFP agreement. It's quite likely they aren't (consumer ISPs would never upload enough to large providers to get a symmetric agreement worth their while.)
Care to link me?
[deleted]
In case you're wondering who wrote this, they are closely connected to Comcast.
From digitalsociety.org: "Digital Society is an independent 501(c)3 non-profit organization, funded by donations from Jon Henke and from Arts+Labs."
From artsandlabs.com: "Arts+Labs partners include entertainment companies, software providers, telecommunications providers, artists and creators committed to delivering innovative and creative digital products to consumers." (including NBC Universal)
From Wikipedia: "As of August 2010, Comcast's proposed acquisition of a majority stake in NBC Universal is pending government approval."
From digitalsociety.org: "Digital Society is an independent 501(c)3 non-profit organization, funded by donations from Jon Henke and from Arts+Labs."
From artsandlabs.com: "Arts+Labs partners include entertainment companies, software providers, telecommunications providers, artists and creators committed to delivering innovative and creative digital products to consumers." (including NBC Universal)
From Wikipedia: "As of August 2010, Comcast's proposed acquisition of a majority stake in NBC Universal is pending government approval."
So by "closely connected" you mean a non-profit that received some funding from an organization that has partnered with many companies, one of which Comcast may buy some stock of?
In case your wondering, a funding paper trail for a non-profit doesn't entail bias in a writer for (or employed by) said organization. Bias in a writer doesn't entail factually incorrect information or invalidate a writer's arguments.
This statement does not contradict anything explicitly stated by the OP, in case you're wondering.
This statement does not contradict anything explicitly stated by the OP, in case you're wondering.
Comcast statement - http://blog.comcast.com/2010/11/comcast-comments-on-level-3....
(also crap IMHO - I am not particularly inclined to believe their claim that Level 3 got same terms as Akamai and other CDNs for non-Netflix content)
(also crap IMHO - I am not particularly inclined to believe their claim that Level 3 got same terms as Akamai and other CDNs for non-Netflix content)
Thanks for the link. I see that the official statement is using much more measured language.
I'm pretty amused by this quote: "what Level 3 wants is to pressure Comcast into accepting more than a twofold increase in the amount of traffic Level 3 delivers onto Comcast's network -- for free."
Isn't Comcast paying Level 3 to deliver traffic to them? If Comcast is not happy with the price Level 3 charges them, can't they switch to a different tier 1 network?
I'm pretty amused by this quote: "what Level 3 wants is to pressure Comcast into accepting more than a twofold increase in the amount of traffic Level 3 delivers onto Comcast's network -- for free."
Isn't Comcast paying Level 3 to deliver traffic to them? If Comcast is not happy with the price Level 3 charges them, can't they switch to a different tier 1 network?
I'm waiting on Level 3 to call out Comcast on that specifically. That would make the argument of Level 3's accusation against Comcast.
The discussion of this article over on reddit has uncovered some interesting past articles by the same author, and it seems he has a strong bias on this topic
http://www.reddit.com/r/technology/comments/edxjp/level_3_ou...
http://www.reddit.com/r/technology/comments/edxjp/level_3_ou...
true! every one of those 'Digital Society' paid promo links on memeorandum has the strong whiff of incumbent telecom corporate shilling. I have never seen one get popular there, and am quite surprised to see it #1 here.
The key to this argument is whether Level 3 will have the same the same terms that Akamai and other CDNs do for non-Netflix content. Comcast claims that is the case, but I am not particularly inclined to believe them.
The key to this argument is whether Level 3 will have the same the same terms that Akamai and other CDNs do for non-Netflix content. Comcast claims that is the case, but I am not particularly inclined to believe them.
My understanding is that Level 3 (and Akamai and Limelight and soon everyone) has to pay Comcast for all traffic.
Personally, I would take three extra network hops and a slower internet if it meant Comcast got screwed. Having been practically forced numerous times to use Comcast as a home internet provider because there were no other options, spent days (when you add it all up) on their stupid support line and having been bit by not calling after 3 or 6 months "trial" (or actually them putting you on hold enough - like 1+ hour), I dont care. This article is asking me to be worried/sorry about the equivalent of a child molester of ISPs. Sorry. NO
"Now it makes perfect sense how Level 3 managed to outbid Akamai since no CDN provider operating legally could outbid hot goods."
I think he means contractually rather than legally, very big difference and suggests that this story is a beat-up. There is no way in the world that Comcast (a dedicated end user provider) was almost equal with L3 (almost dedicated to providing first-tier backbone to private networks and colo)
Peering is interesting in that you can get a lot of free hosting if you find a company that does a lot of inbound but no outbound. We hosted all of the OpenBSD, FreeBSD and 100+ other projects au mirrors for 10+ years for free hitting peaks of gbits+ because of the provider having to make up peering arrangements. I imagine Comcast would be the same (we did it with a similar company - home broadband provider)
This is definitely a case of Comcast attempting to double-dip and feeding a story to a journalist to get their side of the argument out (the journalist even put a hit on one of his competing publications for 'getting it wrong')
I think he means contractually rather than legally, very big difference and suggests that this story is a beat-up. There is no way in the world that Comcast (a dedicated end user provider) was almost equal with L3 (almost dedicated to providing first-tier backbone to private networks and colo)
Peering is interesting in that you can get a lot of free hosting if you find a company that does a lot of inbound but no outbound. We hosted all of the OpenBSD, FreeBSD and 100+ other projects au mirrors for 10+ years for free hitting peaks of gbits+ because of the provider having to make up peering arrangements. I imagine Comcast would be the same (we did it with a similar company - home broadband provider)
This is definitely a case of Comcast attempting to double-dip and feeding a story to a journalist to get their side of the argument out (the journalist even put a hit on one of his competing publications for 'getting it wrong')
Level 3 found itself in Comcast's shoes back in 2005. They felt that Cogent Communications unfairly taxed its network and made the exact same argument that Comcast is making today, and even temporarily pulled the plug on its connection to Cogent, cutting off some parts of the Internet for millions of Cogent customers.
How exactly could an ISP ever match the upload volume of a downstream contract?
It's technically impossible.
Even without netflix, the download to upload ratio is huge.
It's technically impossible.
Even without netflix, the download to upload ratio is huge.
Many (most?) large cable providers, such as Comcast, have co-location and hosting businesses. After all, they have significant server facilities and a serious traffic deficit to make up.
They can't, which why this article is bullshit. Settlement-free peering agreements exist because they are mutually beneficial, not to directly generate revenue. Both benefit from cost savings and Level3 can deliver better service quality, which is a big competitive advantage in their business.
utterly stupid.
comcast is a customer of level3, not a peer. even though marketing has blurred the lines (i blame overly ambitious hosting companies), there is a very large difference between peering and being a transit customer.
what applies here is a transit agreement, even though george ou completely skips over that in his analysis by assuming comcast is not a customer.
to be redundant: this is not a peering spat, because comcast and level3 DO NOT PEER, comcast is a level3 customer.
customer.
customer.
comcast is a customer of level3, not a peer. even though marketing has blurred the lines (i blame overly ambitious hosting companies), there is a very large difference between peering and being a transit customer.
what applies here is a transit agreement, even though george ou completely skips over that in his analysis by assuming comcast is not a customer.
to be redundant: this is not a peering spat, because comcast and level3 DO NOT PEER, comcast is a level3 customer.
customer.
customer.
Right, where comcast pays L3 to deliver what comcast has to send. L3 still needs to pay someone (either comcast or someone else) to take the what they have to offer. No one wants traffic, thats why L3 has to pay to send Netflix to customers.
Usually in a transit relationship the customer pays for the whole pipe; there isn't accounting of incoming vs. outgoing.
> comcast is a customer of level3
It sounds to me like they used to be equals, and now Level3 is the customer.
It sounds to me like they used to be equals, and now Level3 is the customer.
Lots of people seem to be forgetting that content providers want to connect to customers just as much as the customers want the content.
Comcast:"L3 you want to connect to all my cable customers fast and you don't want to pay even more to get to me through AT&T so lets make a deal."
It makes perfect sense that Comcast should be paid for access to their customers.
Comcast:"L3 you want to connect to all my cable customers fast and you don't want to pay even more to get to me through AT&T so lets make a deal."
It makes perfect sense that Comcast should be paid for access to their customers.
"It makes perfect sense that Comcast should be paid for access to their customers."
I disagree, heartily. Comcast's customers are paying Comcast for access to the entire internet, not just to the little segment of the network that Comcast owns.
Exactly this, especially in markets where there isn't consumer choice (which is something I don't see anyone talking about on HN).
Conceivably an ISP could charge customers for a connection AND charge providers for access to that base. For instance, magazines charge for both the subscription and the advertisements.
However, I can decide if I want Harper's or US Weekly. If one provides value and the other doesn't, I get to pick. With ISPs, the same feature isn't at play: I either use Road Runner or I don't get high-speed bandwidth. At that point, ISPs have a larger obligation to protect the customers by not doing things like what Comcast is doing.
Conceivably an ISP could charge customers for a connection AND charge providers for access to that base. For instance, magazines charge for both the subscription and the advertisements.
However, I can decide if I want Harper's or US Weekly. If one provides value and the other doesn't, I get to pick. With ISPs, the same feature isn't at play: I either use Road Runner or I don't get high-speed bandwidth. At that point, ISPs have a larger obligation to protect the customers by not doing things like what Comcast is doing.
And the bullshit part about this is that most of Comcast's customers have little choice in being a customer. They're trapped and have no ability to choose.
Even though Comcast customers already pay Comcast to receive those bytes? As others have pointed out above.
Paid magazines often still sell advertising space, same is true for paid cable channels, so it's not unheard of to charge both the "broadcaster" and the "receiver".
But that is a model in which the advertises is "subsidizing" the effective cost to the end user. (Whether or not it's an actual subsidy or just a nice bit of profit for the cable network/provider is anyone's guess.)
Obviously, without legislation, the internet will end up being no different, with Comcast, AT&T, and ultimately high-traffic sites like espn.com nickel-and-diming each other and end users. Sticking to the principles on which the web was (allegedly?) founded doesn't hold much water in a shareholders meeting.
I hate to sound so cynical and resigned to this fate, but left to their own devices this seems like exactly what the big ISP's and sites will do.
Obviously, without legislation, the internet will end up being no different, with Comcast, AT&T, and ultimately high-traffic sites like espn.com nickel-and-diming each other and end users. Sticking to the principles on which the web was (allegedly?) founded doesn't hold much water in a shareholders meeting.
I hate to sound so cynical and resigned to this fate, but left to their own devices this seems like exactly what the big ISP's and sites will do.
I agree that it is very easy for this to lead to a bad situation for customers. But that subsidizing model still applies. You know how much it costs to connect a cable line to a house? That wiring costs 100s-1000s of dollars per house (why FioS went to apartment buildings first). Of course the end users are getting subsidies.
[deleted]
The article talks about 5 times increase in volume of the traffic.
Wouldn't it mean Comcast gets to make extra bucks even though number
of Comcast customers who access Netflix will not increase that much?
Right or wrong, Comcast wants a cut every time Netflix (or any one
else) changes CDN provider.
[deleted](3)
Paying for transits is easy to understand, and in a complicated topology paying for bytes-in as a peer makes sense because they may be transits.
For a leaf node network like Comcast, whose customers pay to receive bytes and pay to have their connections installed to then demand payment from producers for supplying those bytes seems unjustified.
The author tries to hang this on the relatively higher build out cost per customer Mbps of residential nodes compared large commercial drops, but that is just different businesses. The residential providers already charge many multiples higher for bandwidth, this is just haggling about the multiplier.