Netflix responds to Verizon(scribd.com)
scribd.com
Netflix responds to Verizon
http://www.scribd.com/doc/228871116/Response-to-Demand-Letter
261 comments
Nice jab with (what seems to be) an allusion to the New Jersey bridge-closing scandal that engulfed Gov. Chris Christie: "... like blaming drivers on a bridge for traffic jams when you're the one who decided to leave three lanes closed during rush hour."
I'd only change it to "... blaming drivers on your toll bridge when you're the one ..."
The drivers (customers) are paying for the bridge!
The drivers (customers) are paying for the bridge!
Sure, but they're only using the bridge to drive to McDonalds. It's McDonalds generating all that traffic -- they should pay the bridge operator too! /sarcasm.
If we had usage-paid highways (which we couldn't have earlier due to missing technology but now such things start to pop up), I can imagine some remotely-located mega-store could advertise as "come to us and we'll pay your road toll - both directions!".
It's like asking Walmart to pay for the cost of the welfare that its employees take.
Wait!
Wait!
Was that sarcasm? If not, and it was a serious idea then keep in mind Walmart is not the "parent" of their employees - it's not responsible for their upkeep.
Hopefully that was just sarcasm and I'm replying for no reason.
Hopefully that was just sarcasm and I'm replying for no reason.
They do. Property taxes.
Seeing as ISPs also receive money from the government and Netflix pays taxes, you could make the same pointless observation about Netflix and Verizon.
But you shouldn't. This discussion is about a direct company-to-company attempted shakedown, not inefficient procurement.
But you shouldn't. This discussion is about a direct company-to-company attempted shakedown, not inefficient procurement.
I'm not sure you mean to argue for a publicly funded Internet, but that's basically what this implies.
You've already got that as the government already pays subsidies. I'm all for basic infrastructure to be collectively owned; be it co-ops, collectives, not for profits or even the government (local, national etc). This is the way it is for roads, rubbish, sewerage, etc. If you want a premium / luxury service then feel free to pay extra. Contract out support, maintenance, sales for the infrastructure by all means, but the ownership should be a collective of the users.
Property taxes tied to the level of traffic destined for a particular commercial location? This is the first I've heard of something like this, can you tell me more?
Property taxes for a given location are determined by the size and nature of the business.
Property taxes are based on a poor approximation of what a property would be expected to sell for. The relationship between that and the size or nature of the business operating there is highly attenuated, and it's almost entirely decoupled from how much traffic a business generates on any particular road.
They don't pay taxes to toll road operators.
I'd say it's like blaming Toyota for the traffic jams when you close lanes on the bay bridge.
Or alternatively like planing the SF Giants or AT&T Park for the traffic jams when you close lanes on the bay bridge.
Or alternatively like planing the SF Giants or AT&T Park for the traffic jams when you close lanes on the bay bridge.
Isn't that true with all bridges though? You don't have to pay for each use, but you sure paid for the construction and potentially the upkeep (or are likely to have paid).
This is going very tangential, but no, not all bridges are directly paid for only by people driving on them.
To make public bridges into an ISP analogy, imagine: municipal and state taxes as well as tax revenue from other states (via Federal highway dollars) pay for a municipal broadband network. You may or may not actually use the service, and you pay for it either way. You likely don't have any other choice of ISP. And it's not-for-profit, and the general public and lawmakers constantly clamor and legislate for better service at lower prices.
To make public bridges into an ISP analogy, imagine: municipal and state taxes as well as tax revenue from other states (via Federal highway dollars) pay for a municipal broadband network. You may or may not actually use the service, and you pay for it either way. You likely don't have any other choice of ISP. And it's not-for-profit, and the general public and lawmakers constantly clamor and legislate for better service at lower prices.
Although not true, I always liked to think that taxes on my vehicle, fuel, and tolls paid for the infrastructure upon which I drive. I have no idea whether those are sufficient, but I'll bet in my state they state of high taxes and crappy infrastructure they are more than enough.
Generally, the taxes and tolls levied on drivers only pay for ~50% of US road spending - the balance comes out of the general tax pool on both drivers and non-drivers, so it's (arguably) heavily subsidized. (But one could argue that non-drivers benefit from the road system in terms of things delivered to them by truck, etc.)
Some bridges, such as the Seattle Lake Washington bridge, require a toll to cross even though they're on a toll-free highway.
I don't see the advantage of a Netflix throwing a political jab at Chris Christie when in a statement about something entirely unrelated. It would just serve to make Christie supporters turn on Netflix. As far as I'm aware only 2 lanes were closed and Christie, nor his staff, said anything about blaming the drivers.
Despite the similarities, it seems to me like this was just an analogy, nothing more.
Despite the similarities, it seems to me like this was just an analogy, nothing more.
Christie supporters turn on Netflix? Doubtful.
How about the rest of the nation sees Verizon as a corrupt entity that's willing to screw over it's own constituency for dollars?
How about the rest of the nation sees Verizon as a corrupt entity that's willing to screw over it's own constituency for dollars?
I hope Netflix makes this a standard feature for all ISPs. Bonus point for more transparency, e.g. network graphs comparing peak to off-peak.
This sort of visibility into the real problems means the ISPs can't hide behind their lies.
This sort of visibility into the real problems means the ISPs can't hide behind their lies.
You're very optimistic.
Myself, I think it's far more likely that ISPs are about to successfully run out the clock on all this, kill net neutrality, and then they won't have to ever worry about this or any "transparency" efforts again. Everyone can just pay more for ever-crappier service, and there will be no relevant data to examine, because there will be no "standard" level of service anyone has access to, and therefore no way to compare one level of service to any other level of service at another time or via someone else's product...or at least, no way to do anything about it other than impotently gripe.
Myself, I think it's far more likely that ISPs are about to successfully run out the clock on all this, kill net neutrality, and then they won't have to ever worry about this or any "transparency" efforts again. Everyone can just pay more for ever-crappier service, and there will be no relevant data to examine, because there will be no "standard" level of service anyone has access to, and therefore no way to compare one level of service to any other level of service at another time or via someone else's product...or at least, no way to do anything about it other than impotently gripe.
I find it remarkable that you have an Eeyore, "With Folded Hands" take on the first demonstrably successful action against these last mile ISPs.
Companies like Verizon don't casually threaten legal action, this is really getting to them.
(Perhaps because one of Verizon's unique selling propositions is the quality of their networks, this has the potential for broad damage to their brand.)
Companies like Verizon don't casually threaten legal action, this is really getting to them.
(Perhaps because one of Verizon's unique selling propositions is the quality of their networks, this has the potential for broad damage to their brand.)
Perhaps you're right, and I'm being too pessimistic and harsh about this. But the perspective gained from watching the big ISPs over the last couple decades doesn't make me hopeful.
Why would you say this is a "demonstrably successful" action, btw? Has Verizon actually changed any of their business practices for the better? So far, all I can see is that they are tossing around legal threats. That's not progress; that's Verizon's gigantic legal team gearing up to make sure Verizon can continue to do Absolutely Nothing Positive about the issues in question.
Why would you say this is a "demonstrably successful" action, btw? Has Verizon actually changed any of their business practices for the better? So far, all I can see is that they are tossing around legal threats. That's not progress; that's Verizon's gigantic legal team gearing up to make sure Verizon can continue to do Absolutely Nothing Positive about the issues in question.
Ultimately this is a political battle, and Netflix is demonstrating they can get the ears of their downtrodden subscribers. This is an existential threat to the business model of the big last mile providers save perhaps Century, and might even help push them into the worst case of common carrier status, as the FCC is threatening.
Right now I don't read that threat as credible, but, let's say, how about after the first few Congresscritters are sent home to spend more time with their families after an election? We gun owners, probably less numerous than voting Internet users (don't believe gun ownership self-reporting in surveys, for obvious reasons), with nothing more than our organized votes, have moved mountains at every level of government over the last three decades. Ask ex-Speaker of the House Tom Foley about our power (first Speaker to be turned out of office since the Civil War in 1862).
As the Internet, and good Internet service, gets every more important, the potential is there. Getting back to my analogy, the gun grabbers had a fantastic run for a couple of decades (or more than a century, depending on how you count this and gun control starting to really bite non-minorities), before politically effective resistance started showing results.
Right now I don't read that threat as credible, but, let's say, how about after the first few Congresscritters are sent home to spend more time with their families after an election? We gun owners, probably less numerous than voting Internet users (don't believe gun ownership self-reporting in surveys, for obvious reasons), with nothing more than our organized votes, have moved mountains at every level of government over the last three decades. Ask ex-Speaker of the House Tom Foley about our power (first Speaker to be turned out of office since the Civil War in 1862).
As the Internet, and good Internet service, gets every more important, the potential is there. Getting back to my analogy, the gun grabbers had a fantastic run for a couple of decades (or more than a century, depending on how you count this and gun control starting to really bite non-minorities), before politically effective resistance started showing results.
"Companies like Verizon don't casually threaten legal action, this is really getting to them."
Are you sure? As someone who has dealt with more than 40 different threats of legal action from major companies, I would have guessed that this is more or less their default response.
Are you sure? As someone who has dealt with more than 40 different threats of legal action from major companies, I would have guessed that this is more or less their default response.
Ah, I should clarify, "to those who can fight back effectively at every level".
It also depends on the domain, e.g. Disney protecting IP or ASCAP uncompensated public performances is one thing, and the latter's model can run into PR buzz saws, e.g. Boy and Girl Scouts sings at camps.
Whereas this is part of a big fight that goes right to the heart of a significant part of Verizon's business, and an even more significant part of their potential future business. When Disney threatens a day care center for having an unlicensed depiction of one of their characters, it's routine. Whereas I expect this threatening letter to Netflix was run way up the flagpole; Verizon hasn't typically been that maladroit with PR, have they?
In what domains have you received threats of legal action?
It also depends on the domain, e.g. Disney protecting IP or ASCAP uncompensated public performances is one thing, and the latter's model can run into PR buzz saws, e.g. Boy and Girl Scouts sings at camps.
Whereas this is part of a big fight that goes right to the heart of a significant part of Verizon's business, and an even more significant part of their potential future business. When Disney threatens a day care center for having an unlicensed depiction of one of their characters, it's routine. Whereas I expect this threatening letter to Netflix was run way up the flagpole; Verizon hasn't typically been that maladroit with PR, have they?
In what domains have you received threats of legal action?
I am continually surprised that Netflix has failed to play any real hardball. Seed (ie: astroturf) a class-action lawsuit against Verizon for breach of contract. Make a big PAC donation to the attorney general in a consumer-friendly state like Washington or Massachusetts and let the dogs run. Get the city council of a largish and liberal locale like Seattle to consider a municipal internet service. Heck, cut off service to Verizon subscribers the day House of Cards comes out.
I've always thought that Netflix was in a tough spot when it comes to cutting consumers off, because (1) they'd be losing money, and (2) they'd need to responsibly refund those consumers or else they'll look terrible.
However, TV networks cut off access to cable provider customers when they can't reach a deal on price... so why can't Netflix do the same? I mean, if ISPs really want to turn the Internet into a cable-like place, they should expect that similar disputes are going to happen.
However, TV networks cut off access to cable provider customers when they can't reach a deal on price... so why can't Netflix do the same? I mean, if ISPs really want to turn the Internet into a cable-like place, they should expect that similar disputes are going to happen.
You know... that's not a bad idea. So long as they refund (or more likely extend) the accounts for people accessing Netflix from those ISPs.
Or, maybe Netflix should intentionally limit the number of people that could access their service from a given ISP so that the overall quality is maintained? If you try to access the service and you get denied for ISP capacity reasons, then you'd get that day refunded.
They need to do something to make sure that the customer knows where the problem is. But, instead of playing too rough, I think publicly shaming the ISPs is the best approach. At least for now.
Or, maybe Netflix should intentionally limit the number of people that could access their service from a given ISP so that the overall quality is maintained? If you try to access the service and you get denied for ISP capacity reasons, then you'd get that day refunded.
They need to do something to make sure that the customer knows where the problem is. But, instead of playing too rough, I think publicly shaming the ISPs is the best approach. At least for now.
Something akin to the cable companies cutting channels until deals are met might be blocking certain shows only (netflix originals for instance). As long as there was a warning when you signed up I suppose (certain networks may not be able to stream netflix originals).
I don't know, this strikes me as "real" hardball. Many of the things you suggest are also that, but ultimately this is a political problem and will be won or lost in the political arena. And Netflix is getting a lot of voters pissed off at Verizon, and no doubt other ISPs in the future.
Some of the things you suggest will affect politicians, but the ultimate one is a credible threat of sending them home to spend more time with their families after an election. It's how gun owners have moved mountains starting in the mid-80s or so (the area I'm most familiar with), and if Netflix is playing the long game, it's how they're going to win ... and overall help us win the net neutrality battle.
Some of the things you suggest will affect politicians, but the ultimate one is a credible threat of sending them home to spend more time with their families after an election. It's how gun owners have moved mountains starting in the mid-80s or so (the area I'm most familiar with), and if Netflix is playing the long game, it's how they're going to win ... and overall help us win the net neutrality battle.
>Heck, cut off service to Verizon subscribers the day House of Cards comes out.
They'd be shooting themselves in the foot. That will just make Verizon subscribers mad.
They should just flash up messages every so often when the network is congested to get people to call their congresspeople and ask them to support net neutrality and firing ex-comcast lobbyists from the FCC.
They'd be shooting themselves in the foot. That will just make Verizon subscribers mad.
They should just flash up messages every so often when the network is congested to get people to call their congresspeople and ask them to support net neutrality and firing ex-comcast lobbyists from the FCC.
Viacom and CableOne are fighting it out. Negotiations for licensing content failed.
Viacom is now blocking CableOne customers from accessing videos (e.g. Comedy Central) online.
I've heard Viacom is encouraging customers to contact CableOne about the problem.
Viacom is now blocking CableOne customers from accessing videos (e.g. Comedy Central) online.
I've heard Viacom is encouraging customers to contact CableOne about the problem.
[deleted]
I'm not too sure. Netflix has one hell of a popular product. And that gives them leverage. The ISPs have a product too, but it's nowhere near as good as other countries around the world.
When you have a popular product, you have customers on your side, and that's a huge boon for Netflix.
When you have a popular product, you have customers on your side, and that's a huge boon for Netflix.
I agree that our bandwidth to price ratio is shitty in this country. But until we have viable competition in THIS country, our rates won't come down. John Oliver's rant on Net Neutrality (ironically on HBO) sums this up pretty nicely: https://www.youtube.com/watch?v=fpbOEoRrHyU
When you say "bandwidth to price ratio is shitty in this country," are you in a country where you pay $79.95/month for 250GB quota on ADSL2+ which connects at 6Mbps?
http://www.iinet.net.au/internet/broadband/adsl/
http://www.iinet.net.au/internet/broadband/adsl/
Effectively, yes.
We're in a country where we pay $50/month for 150GB quota on ADSL2+ which connects at 5Mbps.
There's a $10/50GB overage charge. So if we were to hit the quota you have, we'd be paying roughly the same ($70/month). Note that the price listed does not include taxes and fees, which bump it up to within a few dollars of your $79 price point.
http://www.att.com/shop/en/internet/internet-service.html
We're in a country where we pay $50/month for 150GB quota on ADSL2+ which connects at 5Mbps.
There's a $10/50GB overage charge. So if we were to hit the quota you have, we'd be paying roughly the same ($70/month). Note that the price listed does not include taxes and fees, which bump it up to within a few dollars of your $79 price point.
http://www.att.com/shop/en/internet/internet-service.html
It sounds like they might:
"We are testing this type of messaging across the U.S. with multiple providers" [end of 1st paragraph]
and
"The current transparency test ... is scheduled to end June 16 and we are evaluating rolling it out more broadly. Regardless of this specific test, we will continue to work on ways to communicate network conditions to our customers." [last paragraph]
"We are testing this type of messaging across the U.S. with multiple providers" [end of 1st paragraph]
and
"The current transparency test ... is scheduled to end June 16 and we are evaluating rolling it out more broadly. Regardless of this specific test, we will continue to work on ways to communicate network conditions to our customers." [last paragraph]
Internet current conditions. Love it. "And it looks like we're going to have a very blustery afternoon making for a tough easterly traffic situation and a high of 97 cuils"
For those who may not have gotten the reference, I've cuil'd it for you: https://en.wikipedia.org/wiki/Cuil
|This sort of visibility into the real problems means the ISPs can't hide behind their lies.|
Unfortunately, all the transparency in the world can't effect change when the consumer lacks choice in the marketplace.
ISPs need to be classified as common carriers.
Unfortunately, all the transparency in the world can't effect change when the consumer lacks choice in the marketplace.
ISPs need to be classified as common carriers.
Transparency alone never effects change. But angering the voter can pressure the congresscritter can put the fear of regulation into a company.
I'll admit I don't see much change coming from this individual spat, but I find it reassuring that there's at least this means of "recourse" for content providers.
I'll admit I don't see much change coming from this individual spat, but I find it reassuring that there's at least this means of "recourse" for content providers.
What is the messaging they're displaying to Verizon customers?
"The Verizon network is crowded right now." https://twitter.com/yurivictor/status/473978204852453376
Instead of the normal "buffering" message, they were apparently displaying something to the effect of "sorry, it looks like Verizon's network is congested" for Verizon customers.
Verizon network is crowded right now. Adjusting video for smoother playback.
Should be "degrading image for smoother playback". We know that's what's happening; do most people?
Even if they don't, they will be annoyed enough at seeing the message that says they have to wait.
> ISPs can't hide behind their lies.
NetFlix isn't exactly being honest here either. They could easily delay the start of video playback to allow for more caching on congested networks to offer their customers a better experience. They could also allow customers to queue videos to be downloaded/cached locally in advance. These things would provide a better experience to NetFlix's customers so why aren't they doing them? I suppose they cost too much to develop or would increase NetFlix's costs with content providers. In other words NetFlix is trying to maximize their own profits and throwing a fit that ISPs are doing the exact same thing.
NetFlix isn't exactly being honest here either. They could easily delay the start of video playback to allow for more caching on congested networks to offer their customers a better experience. They could also allow customers to queue videos to be downloaded/cached locally in advance. These things would provide a better experience to NetFlix's customers so why aren't they doing them? I suppose they cost too much to develop or would increase NetFlix's costs with content providers. In other words NetFlix is trying to maximize their own profits and throwing a fit that ISPs are doing the exact same thing.
That doesn't work. Such a delay and a couple of other tricks only help against jitter. If you're short of bandwidth for tens of seconds or longer, the only solution is to stream at very low bitrate.
(FWIW, I've worked for a streaming company and one of my projects there involved bandwidth measurement.)
(FWIW, I've worked for a streaming company and one of my projects there involved bandwidth measurement.)
I believe the core problem here is that ISPs and infrastructure are owned by the same companies. Based on evidence from other countries, like Japan, these problems would largely go away if it was illegal to own both the infrastructure and provide service to customers.
Is it possible that anti-trust laws could be used to force these large ISPs to break up into separate companies, one that owns and provides the network and the other that only leases the lines and provides service?
Such a breakup would allow new, smaller, ISPs to leverage the infrastructure while provided superior service and potentially better pricing models.
Of course there would still be the problem of competing infrastructure companies. Some of that could be alleviated by increasing competition through public/private fiber installation projects.
Is it possible that anti-trust laws could be used to force these large ISPs to break up into separate companies, one that owns and provides the network and the other that only leases the lines and provides service?
Such a breakup would allow new, smaller, ISPs to leverage the infrastructure while provided superior service and potentially better pricing models.
Of course there would still be the problem of competing infrastructure companies. Some of that could be alleviated by increasing competition through public/private fiber installation projects.
>I believe the core problem here is that ISPs and infrastructure are owned by the same companies.
Exactly. MSO (Cable) & Telco ISPs that offer video products are already prioritizing their own VoD (Video-on-Demand) offerings over data to ensure QoS (quality of service). Whether by over-provisioning the virtual circuits to their customers, not amortizing the data consumed against a hard or soft cap limit, or directly peering the VoD servers with the edge routers, ISP video gets the priority because that is where the big profits are. There is no value proposition for BIG ISP to provide free peering with their competition.
Exactly. MSO (Cable) & Telco ISPs that offer video products are already prioritizing their own VoD (Video-on-Demand) offerings over data to ensure QoS (quality of service). Whether by over-provisioning the virtual circuits to their customers, not amortizing the data consumed against a hard or soft cap limit, or directly peering the VoD servers with the edge routers, ISP video gets the priority because that is where the big profits are. There is no value proposition for BIG ISP to provide free peering with their competition.
It should also be noted, that this makes perfect sense with regards to the way the telcos see the world - selling discrete well-provisioned services. This is the classic datagrams versus circuits battle, still burning strong decades letter. And of course that's because this is where the money is - differentiated services lead to price discrimination and high profit margins, while commodity bits are a highly competitive race to the bottom. And as computing power keeps dropping in price, they both require ever-similar infrastructure buildout (contrary to "End-to-End Arguments in System Design")
yes, that would probably work, but they have such a huge influence on washington that at the moment we're having a hard time even keeping net neutrality alive.http://apps.fcc.gov/ecfs/upload/display?z=1xne7
I believe the core problem here is that ISPs and infrastructure are owned by the same companies. Based on evidence from other countries, like Japan, these problems would largely go away if it was illegal to own both the infrastructure and provide service to customers.
That is certainly an idea worth exploring, but I'm not sure it's an automatic win.
We could consider European railway networks as a somewhat analogous situation. In several countries, one business or public entity owns and maintains the tracks and related infrastructure, while one or more others own and operate the trains themselves.
We have a better service here in the UK today with that model than we used to with the nationalised all-in British Rail, in some respects at least. On the other hand, the costs of that system are crazy, both in ticket prices and government subsidy, so it's far from clear whether the improvements have come because of the split in responsibilities or despite it.
We're not the only ones whose system doesn't always work, though. Every now and then, you see a real howler due to miscommunication, as IIRC the French had recently when it turned out that the super new trains wouldn't be able to fit next to the platforms at a load of older stations.
It seems to me that the main problem with ISPs in the US is not so much being both infrastructure provider and service provider as it is having little if any real competition on either count. Rather like the banking system, the US appears to be maybe a decade behind most of the developed world in this respect, for no particularly good reason. I can only assume it's a consequence of politics and/or questionable business environment.
That is certainly an idea worth exploring, but I'm not sure it's an automatic win.
We could consider European railway networks as a somewhat analogous situation. In several countries, one business or public entity owns and maintains the tracks and related infrastructure, while one or more others own and operate the trains themselves.
We have a better service here in the UK today with that model than we used to with the nationalised all-in British Rail, in some respects at least. On the other hand, the costs of that system are crazy, both in ticket prices and government subsidy, so it's far from clear whether the improvements have come because of the split in responsibilities or despite it.
We're not the only ones whose system doesn't always work, though. Every now and then, you see a real howler due to miscommunication, as IIRC the French had recently when it turned out that the super new trains wouldn't be able to fit next to the platforms at a load of older stations.
It seems to me that the main problem with ISPs in the US is not so much being both infrastructure provider and service provider as it is having little if any real competition on either count. Rather like the banking system, the US appears to be maybe a decade behind most of the developed world in this respect, for no particularly good reason. I can only assume it's a consequence of politics and/or questionable business environment.
> as IIRC the French had recently when it turned out that the super new trains wouldn't be able to fit next to the platforms at a load of older stations.
So they made it through the provisioning process, did they make it all the way to a... test run? :)
So they made it through the provisioning process, did they make it all the way to a... test run? :)
Nahhh, it was a non-story actually. The company making the new trains had actually gone out and measured the stations before designing the trains. They had then notified the entity responsible for the stations, letting them know that the stations would need to be reworked. Work has actually already started on some stations, but is running behind schedule and not all stations will be ready when the resins come on line. This entity has tried to blame their scheduling mismanagement on the train manufacturer, but as there are written agreements to do the work on the stations, that dig won't hunt.
Just make sure you do it properly the first time, and don't trust the telcos when they tell you they'll "self regulate". Otherwise you'll end up with the mess that Australia has, which led to the previous Government setting up the NBN project.
Not only that, but any time a retail ISP submits complaints about a customer's service, Telstra calls the customer directly and tells them that their problems will go away if they subscribe to a Telstra plan (and the problems do go away, strangely enough).
So make sure there is a proper separation of the infrastructure and retail portions of the telco behemoth right from day 1. Otherwise you'll be dealing with a situation that is worse than what you have now.
Not only that, but any time a retail ISP submits complaints about a customer's service, Telstra calls the customer directly and tells them that their problems will go away if they subscribe to a Telstra plan (and the problems do go away, strangely enough).
So make sure there is a proper separation of the infrastructure and retail portions of the telco behemoth right from day 1. Otherwise you'll be dealing with a situation that is worse than what you have now.
Don't worry, the current government will make the situation so bad that America's telcos will look like the gold standard by comparison.
It's really strange. We're quite far ahead with mobile/cell/wireless coverage and plans but very very far behind in high-speed broadband.
It's really strange. We're quite far ahead with mobile/cell/wireless coverage and plans but very very far behind in high-speed broadband.
So the DSL infrastructure?
>So the DSL infrastructure?
It's not so much about the last mile anymore. Since most customers want bundled voice/data/video, it's about linear video channels and that means content. The big players are always going to have more clout in negotiating the best rates for their bundled channels because they have the most subs (subscribers). Even if FCC mandated unbundling of the CableCos' HFC plant and the Telcos' FTTx plant, no one will be able to compete with the big boys on content.
Also, alot of these plants were not designed for interoperability with other vendors' equipment and troubleshooting would be difficult to impossible. It's no longer as simple as a 2-wire local loop (twisted pair) that you can break out from a split bank to your own frame and DSLAM. The technologies share the same physical media -- think bus topology like a token ring. One faulty ONT in a B-PON or G-PON network can impact service for ~30 customers.
What the FCC could do is mandate the reselling of data services like UNE-P with POTS, and the reseller (e.g. Netflix) could then bundle their own pure VoD product for video over the carrier's data platform So, deregulation (unbundling - platform) is one option.
Otherwise, there would have to be a mandated and fair policy for exchange data between ISPs either at free or at tariffed rates across the board, just like voice trunks.
Data is the new voice and the FCC needs to weigh in and start (de)/regulating or this is what the internet is going to look like 10 years from now.
a.pomf.se/qtkuqv.jpg
It's not so much about the last mile anymore. Since most customers want bundled voice/data/video, it's about linear video channels and that means content. The big players are always going to have more clout in negotiating the best rates for their bundled channels because they have the most subs (subscribers). Even if FCC mandated unbundling of the CableCos' HFC plant and the Telcos' FTTx plant, no one will be able to compete with the big boys on content.
Also, alot of these plants were not designed for interoperability with other vendors' equipment and troubleshooting would be difficult to impossible. It's no longer as simple as a 2-wire local loop (twisted pair) that you can break out from a split bank to your own frame and DSLAM. The technologies share the same physical media -- think bus topology like a token ring. One faulty ONT in a B-PON or G-PON network can impact service for ~30 customers.
What the FCC could do is mandate the reselling of data services like UNE-P with POTS, and the reseller (e.g. Netflix) could then bundle their own pure VoD product for video over the carrier's data platform So, deregulation (unbundling - platform) is one option.
Otherwise, there would have to be a mandated and fair policy for exchange data between ISPs either at free or at tariffed rates across the board, just like voice trunks.
Data is the new voice and the FCC needs to weigh in and start (de)/regulating or this is what the internet is going to look like 10 years from now.
a.pomf.se/qtkuqv.jpg
That JPG is one of the worst arguments on the Internet. It's popular because it works up the rage of "this might happen" netizens, but "might" has been reduced to "doesn't violate the laws of physics."
But, it's not just FUD. https://secure.dslreports.com/forum/r20614148- Taxpayer subsidies helped build the telco network, not dime one of which has been repaid. Meanwhile, they https://secure.dslreports.com/shownews/ATT-Study-We-Should-K... rip out the copper because it's no longer as profitable (as a regulated service) as video and wireless. Cablecos that have a captive market raise their rates as much as their consumer base will bear. As soon as carriers can find a way to monetize something, they won't stop. Value-added services, fees, rate-hikes. It's rent-seeking behavior, like the railroad tycoons. Cable & telco greed is practically a law of physics. That's why we have regulated service for electricity and telephone, because we recognize how essential it is to modern life. And it should be no different for internet.
Even if I bought all your negative emotions about ISPs[1], none of your scare scenarios are at all about the ISP charging more money to the customer depending on what websites they go to.[2]
Even right now, with whatever asshattery Verizon is doing, they haven't even peeped one single word about making the customers of theirs that use Netflix pay more money to Verizon.
"These people do bad things, so you should believe these other bad things I say they might someday do." Lots of people have made very good careers out of enraging the rabble that way.
[1] Your first link uses a bunch of Google searches as evidence, so I can't be 100% sure what it's talking about. But there's a decent chance it's this: https://news.ycombinator.com/item?id=7709910 forex
[2] The second article is about AT&T wanting to get the hell out of the wired communication business. Investors hate the wired business. One way or another you have to pay for that infrastructure, and talking about applying utility-level regulation to the companies eager to leave is not going to draw in new dollars.
Even right now, with whatever asshattery Verizon is doing, they haven't even peeped one single word about making the customers of theirs that use Netflix pay more money to Verizon.
"These people do bad things, so you should believe these other bad things I say they might someday do." Lots of people have made very good careers out of enraging the rabble that way.
[1] Your first link uses a bunch of Google searches as evidence, so I can't be 100% sure what it's talking about. But there's a decent chance it's this: https://news.ycombinator.com/item?id=7709910 forex
[2] The second article is about AT&T wanting to get the hell out of the wired communication business. Investors hate the wired business. One way or another you have to pay for that infrastructure, and talking about applying utility-level regulation to the companies eager to leave is not going to draw in new dollars.
It isn't that the ISPs will force their customers to pay more for access to third-party services, but rather that they will selectively manage their network so that certain third-party services will have sub-optimal data rates. I'm sure that Verizon would rather those customers use their PPV service (or another more-favored service) rather than use Netflix. And because Verizon is making it more painful for people to use Netflix, that is likely happening. This entire thread started out talking about how Netflix is informing their mutual customers where exactly the blame for their degraded service should fall.
To use Comcast as an example, now Netflix is paying Comcast for a connection to their customers as opposed to Cogent (which should have had good bandwidth between them and Comcast). As a result, Netflix will have to charge their customers more. So, don't think that just because the ISPs won't be charging doesn't mean that the customers won't have to pay more.
Now, you could make an argument that it's good that Netflix customers will have to pay more for their service because they were clogging the pipes for everyone else. The problem is that this is a very slippery slope and could lead to a balkanization of the Internet, very similar to the tiered access presented in the JPG you so adamantly disagreed with. What happens when your ISP (because you only have one choice) doesn't have a peering agreement with the network hosting Service X? Sure, you could use Service X, but your experience wouldn't be very good. But good news for you! They do have an agreement with Service Y that will provide you with almost the same data as Service X, but it costs a little more. Did I mention that Service Y kicks back 20% of your monthly service fee back to your ISP for "hosting"? There are plenty of ways that the ISPs can get more of your money. Not all of it involves getting it from you directly.
To use Comcast as an example, now Netflix is paying Comcast for a connection to their customers as opposed to Cogent (which should have had good bandwidth between them and Comcast). As a result, Netflix will have to charge their customers more. So, don't think that just because the ISPs won't be charging doesn't mean that the customers won't have to pay more.
Now, you could make an argument that it's good that Netflix customers will have to pay more for their service because they were clogging the pipes for everyone else. The problem is that this is a very slippery slope and could lead to a balkanization of the Internet, very similar to the tiered access presented in the JPG you so adamantly disagreed with. What happens when your ISP (because you only have one choice) doesn't have a peering agreement with the network hosting Service X? Sure, you could use Service X, but your experience wouldn't be very good. But good news for you! They do have an agreement with Service Y that will provide you with almost the same data as Service X, but it costs a little more. Did I mention that Service Y kicks back 20% of your monthly service fee back to your ISP for "hosting"? There are plenty of ways that the ISPs can get more of your money. Not all of it involves getting it from you directly.
The reason that asinine JPEG gets shared so much is because it makes people rage because they think they'll have to sign up ahead of time for any website they might want to visit. If it were accurate -- if it said "$2 of your Netflix subscription goes to paying ISP interconnection costs," people would shrug and say "well, of course Netflix has ISP costs."
Hey, Netflix could charge more to customers of certain ISPs, too. What a nightmare! If I make a JPEG showing that will everyone go nuts and share it on the Facebook and demand Netflix stop fucking with net neutrality? I mean, Netflix might do it.
Hey, Netflix could charge more to customers of certain ISPs, too. What a nightmare! If I make a JPEG showing that will everyone go nuts and share it on the Facebook and demand Netflix stop fucking with net neutrality? I mean, Netflix might do it.
People who are normally getting <500Kbit connection on a Verizon DSL can get the full 3Mbit connection over a VPN during the same time of day. So, is it that Verizon does not have the capacity, or is it that they are specifically throttling your connection to Netflix? These last miles ISPs get all kinds of benefits for their wire, they get tax benefits, and protection. For instance, they have sued in many cities to keep Google Fiber or other municipal fiber networks out.
I have no idea how this isn't an antitrust case yet. Also, considering that most cable packages cost $80+, and Netflix costs $7, what makes you think that the ISPs will stop at charging Netfix just a few dollars extra per customer. Don't you get it, they don't want to get Netflix to pay them more money, they want to drive Netflix out of business.
I think more so than Netflix, Google has a stake in this. Block youtube and gmail for one month on any ISP that does not sign a Net Neutrality pledge, and see what happens. If black mail is good for the goose, it's good for the gander.
I have no idea how this isn't an antitrust case yet. Also, considering that most cable packages cost $80+, and Netflix costs $7, what makes you think that the ISPs will stop at charging Netfix just a few dollars extra per customer. Don't you get it, they don't want to get Netflix to pay them more money, they want to drive Netflix out of business.
I think more so than Netflix, Google has a stake in this. Block youtube and gmail for one month on any ISP that does not sign a Net Neutrality pledge, and see what happens. If black mail is good for the goose, it's good for the gander.
You make big claims about what other people perceive. This is a straw-man. Clearly it makes /you/ rage, but I suspect it is because it threatens you with cognitive dissidence.
I've experienced this with other MSO engineers. Some MSO engineers even argue that charging a content provider to access their mutual customers is not a rent seeking behavior.
I've experienced this with other MSO engineers. Some MSO engineers even argue that charging a content provider to access their mutual customers is not a rent seeking behavior.
>they haven't even peeped one single word about making the customers of theirs that use Netflix pay more money to Verizon.
Not directly, but if the ISPs charge Netflix a fee that forces Netflix to raise prices for its customers to re-coup those costs (or hypothetically pass it as a surcharge on to its customers on a given ISP) they've essentially taken that money from their customers who also use Netflix. That the money went through an extra set of hands first is a mis-direct. That non-Verizon Netflix customers might be impacted ends up getting cancelled out if most of the major ISPs do the same thing. If Netflix doesn't charge a fee to re-coup those costs, then Netflix has less money to make rights agreements with, and I've got less content to watch. No matter what, I as a Verizon/Netflix customer lose.
Not directly, but if the ISPs charge Netflix a fee that forces Netflix to raise prices for its customers to re-coup those costs (or hypothetically pass it as a surcharge on to its customers on a given ISP) they've essentially taken that money from their customers who also use Netflix. That the money went through an extra set of hands first is a mis-direct. That non-Verizon Netflix customers might be impacted ends up getting cancelled out if most of the major ISPs do the same thing. If Netflix doesn't charge a fee to re-coup those costs, then Netflix has less money to make rights agreements with, and I've got less content to watch. No matter what, I as a Verizon/Netflix customer lose.
You can't hear the fast lane/slow lane (or the hilarious fast lane and faster lane counter from the ISP's) and not see that they are indeed aiming for a tiered system can you?
Consumer ISPs already sell different speeds to different customers.
I understand that it's easy for you to believe that people who have done things you don't like in the past might do entirely different things you don't like in the future, because they do things you don't like.
But for all the ISPs sins, they have never taken even one step down this particular road.
I understand that it's easy for you to believe that people who have done things you don't like in the past might do entirely different things you don't like in the future, because they do things you don't like.
But for all the ISPs sins, they have never taken even one step down this particular road.
>Consumer ISPs already sell different speeds to different customers.
That's not what the fast lane/slow lane is. Fast lane/slow lane is you pay for 50/5 BUT there is an extra charge (on Netflix for example) for you to get Netflix at full speed. Otherwise it's throttled all to hell.
Basically all that image missed was the ISP's going after the content providers instead of adding fees to their existing users.
http://arstechnica.com/tech-policy/2014/04/the-fccs-fast-lan...
That's not what the fast lane/slow lane is. Fast lane/slow lane is you pay for 50/5 BUT there is an extra charge (on Netflix for example) for you to get Netflix at full speed. Otherwise it's throttled all to hell.
Basically all that image missed was the ISP's going after the content providers instead of adding fees to their existing users.
http://arstechnica.com/tech-policy/2014/04/the-fccs-fast-lan...
Companies pay for faster Internet connections all the time. That's Akamai's entire business.
Nothing about this is having the ISP's customer pay more to the ISP based on which websites they want to go to.
Nothing about this is having the ISP's customer pay more to the ISP based on which websites they want to go to.
Everyone is paying for Internet access already. Customers are ostensibly paying for a certain amount of bandwidth, Netflix is providing for a certain amount of bandwidth. ISPs now want more money because "Netflix is generating so much traffic". Your point is essentially that there is no indication they would be asking consumers for more money to use Netflix, except they are doing something even slimier which is throttling Netflix and threatening to sue Netflix for revealing whose fault it is that the customer is not getting what they pay for. Instead of charging more for a specific service they essentially make it impossible to use in their monopolized customer base and hold up their hands like they're doing nothing.
Given the ugliness of what they are doing, the only reason I agree they probably wouldn't charge for tiered service is because they probably don't yet have the balls to take that to the court of public opinion, but in their heart of hearts I'm sure they'd love to triple dip by trolling (in the under-the-bridge sense) Netflix and also charging extra to consumers. If they don't get smacked down now I fully believe they'll take it there.
The disgusting thing is they pretend like they own the Internet rather than acknowledging that Internet only works, exists, and has created their market because of peering agreements. The minute networks start trying to nickel and dime each other the whole thing unravels. God we need to get more lobbyists on the right side of this issue into Washington, and hopefully a fewer congresspeople who were born after the invention of color television.
Given the ugliness of what they are doing, the only reason I agree they probably wouldn't charge for tiered service is because they probably don't yet have the balls to take that to the court of public opinion, but in their heart of hearts I'm sure they'd love to triple dip by trolling (in the under-the-bridge sense) Netflix and also charging extra to consumers. If they don't get smacked down now I fully believe they'll take it there.
The disgusting thing is they pretend like they own the Internet rather than acknowledging that Internet only works, exists, and has created their market because of peering agreements. The minute networks start trying to nickel and dime each other the whole thing unravels. God we need to get more lobbyists on the right side of this issue into Washington, and hopefully a fewer congresspeople who were born after the invention of color television.
> Companies pay for faster Internet connections all the time. That's Akamai's entire business
Akamai doesn't pay for faster connections to ISPs. They pay for more servers located at more places in the network, so that the average network distance from an Akamai server to a user is smaller, using the connections that already exist.
> Nothing about this is having the ISP's customer pay more to the ISP based on which websites they want to go to
No, it's about websites or web services paying more to ISPs so that customers can get to them at full speed instead of being throttled back, even though the website or web service is already paying to have their content on servers spread around the world so it is closer to customers.
Akamai doesn't pay for faster connections to ISPs. They pay for more servers located at more places in the network, so that the average network distance from an Akamai server to a user is smaller, using the connections that already exist.
> Nothing about this is having the ISP's customer pay more to the ISP based on which websites they want to go to
No, it's about websites or web services paying more to ISPs so that customers can get to them at full speed instead of being throttled back, even though the website or web service is already paying to have their content on servers spread around the world so it is closer to customers.
Now you're just being deliberately obtuse. Which ISP do you work for btw? I'm going to guess Comcast.
> Akamai's entire business
Akamai's business is not charging company Y extra to not get throttled by Comcast and you know it.
> Nothing about this is having the ISP's customer pay more to the ISP based on which websites they want to go to.
And now you continue to harp on this specific angle. As I said, all that graphic missed was individual ISP's going after content providers specifically instead of (further) screwing their customers. But it is still creating a tiered experience.
> Akamai's entire business
Akamai's business is not charging company Y extra to not get throttled by Comcast and you know it.
> Nothing about this is having the ISP's customer pay more to the ISP based on which websites they want to go to.
And now you continue to harp on this specific angle. As I said, all that graphic missed was individual ISP's going after content providers specifically instead of (further) screwing their customers. But it is still creating a tiered experience.
Which ISP do you work for btw? I'm going to guess Comcast.
This kind of baseless insinuation is not welcome on hn.
This kind of baseless insinuation is not welcome on hn.
they have never taken even one step down this particular road
Because they've been stopped from doing so? Isn't that what this whole fight over Net Neutrality is about?
because they do things you don't like
It's not just a random pattern of dislike. It's a specific monopolistic pattern of market control that content distributors have demonstrated over and over. How stupid would we have to be to not see where they would like to take this?
I'm normally arguing with folks on HN from my pro-capitalist perspective, but treating monopolies like the major ISPs as players in a free market is a huge mistake.
Because they've been stopped from doing so? Isn't that what this whole fight over Net Neutrality is about?
because they do things you don't like
It's not just a random pattern of dislike. It's a specific monopolistic pattern of market control that content distributors have demonstrated over and over. How stupid would we have to be to not see where they would like to take this?
I'm normally arguing with folks on HN from my pro-capitalist perspective, but treating monopolies like the major ISPs as players in a free market is a huge mistake.
Because they've been stopped from doing so?
This is straight out of the rabble-rousers' playbook when questioned "hey, before you get us worked up about the next disaster our enemies will bring out, why don't you tell us why the disaster you predicted last year didn't happen?"
"Well, it's because we fought so hard against it!! Come on, how stupid do you have to be to not see this? Look at this next tragedy they are going to do, of course it was only because of our brave work stopping them that they couldn't continue." Then link to a cable comedy host agreeing with them as proof they were right.
This is straight out of the rabble-rousers' playbook when questioned "hey, before you get us worked up about the next disaster our enemies will bring out, why don't you tell us why the disaster you predicted last year didn't happen?"
"Well, it's because we fought so hard against it!! Come on, how stupid do you have to be to not see this? Look at this next tragedy they are going to do, of course it was only because of our brave work stopping them that they couldn't continue." Then link to a cable comedy host agreeing with them as proof they were right.
All you have to do to make that image accurate is replace the names of popular web sites with the names of big ISPs.
This is pretty much what Netflix is faced with at the moment. They pay $X for basic internet access, and then pay an additional $Y to access Comcast, $Z to access Verizon, etc.
The ISPs are just making the service slow rather than blocking it, but it doesn't change the fundamentals.
This is pretty much what Netflix is faced with at the moment. They pay $X for basic internet access, and then pay an additional $Y to access Comcast, $Z to access Verizon, etc.
The ISPs are just making the service slow rather than blocking it, but it doesn't change the fundamentals.
This is exactly the model that Comcast would like to see in effect. It's exactly how they manage their cable television service and they've been extremely resistant to even allowing customers to choose channels a la carte. Let's face it, this model works well for them. They aren't arguing with the FCC to change it. They're arguing with the FCC to change how they're allowed to manage their Internet traffic.
"Comcast is bad people, and does bad things, so believe me about these bad things I say they want to do, even though they've never talked about it."
Surely the ISPs have done enough actual bad things that we don't have to imagine bad things they might someday do.
Surely the ISPs have done enough actual bad things that we don't have to imagine bad things they might someday do.
I have no idea what you're talking about. Comcast is already pushing for the ability to create a multi-tiered Internet.
That fact is even splashing over into popular culture, so I'm not sure which set of facts you're dealing with:
http://www.slate.com/articles/technology/future_tense/2014/0...
That fact is even splashing over into popular culture, so I'm not sure which set of facts you're dealing with:
http://www.slate.com/articles/technology/future_tense/2014/0...
There are several different forms that a multi-tiered Internet may take, and the differences matter. Charging customers to access content (as in that fictional strawman image) is not the same as charging content providers to access customers (which is actually happening).
Money is fungible. Costs to service providers /must/ be passed to those paying bills. There is no logical difference between costs to content providers and direct charges to customers.
The argument provided is an illustration to demonstrate who is paying in the end.
The argument provided is an illustration to demonstrate who is paying in the end.
If a comedian repeats it, it must be true? K. . .
Seriously, though, when has Comcast indicated it wants to charge its ISP customers per-website? When has any ISP ever put a surcharge on someone connecting to certain websites?
Seriously, though, when has Comcast indicated it wants to charge its ISP customers per-website? When has any ISP ever put a surcharge on someone connecting to certain websites?
I had the option of putting an LA times link and several others that said the same thing, but the John Oliver link has the same information, draws conclusions without my having to repeat them, and is funnier.
Fact is that Comcast is lobbying big time to allow multi-tiered Internet access. That's public, easily-googleable information, so I don't understand why you're arguing otherwise.
Fact is that Comcast is lobbying big time to allow multi-tiered Internet access. That's public, easily-googleable information, so I don't understand why you're arguing otherwise.
I suspect that you are getting "multi-tiered" out of that article, and then want me to get to that JPEG based on that.
>Since most customers want bundled voice/data/video
I don't think that's what they want. That what they are offered.
I don't think that's what they want. That what they are offered.
I hope Netflix doesn't give in, and calls Verizon's bluff. It's highly unlikely Verizon will sue Netflix, and even if it does just because they are so angry with Netflix over it, it will most likely come out in favor of Netflix, once the Court orders Verizon to show what's really happening behind the scenes and who's fault really is.
In fact, from what I've noticed, Verizon has already lowered their tone about this, and is backing away from threatening Netflix with the lawsuit. So carry on!
These ISP's promised good service to their paying customers, regardless of the conditions. It's their responsibility to live up to those expectations.
In fact, from what I've noticed, Verizon has already lowered their tone about this, and is backing away from threatening Netflix with the lawsuit. So carry on!
These ISP's promised good service to their paying customers, regardless of the conditions. It's their responsibility to live up to those expectations.
Someone on Reddit pointed out that it would be a huge PR win for Netflix if Verizon did sue and they got the right documents in discovery. Depending on the scope maybe they could possibly get things as wide ranging as actual customer speeds as well as the peering points/capacity etc.
Written to Verizon but written for everyone but Verizon. This is a PR battle started by Verizon but I think Netflix just went up 3 - 0.
Netflix lands more squarely on the customer's side on this topic. They have everything to gain from PR coverage and Verizon everything to lose. Seems like a bad move to me that Verizon decided to boost the awareness of this issue in the first place. Perhaps that wasn't the intention, but it was the effect.
ya, was just thinking how non-lawyer-cat-speak the letter is
Ugh. Good for Netflix. Any consumer who doesn't have a strong opinion in favor of net neutrality should read this sort of thing. It's hard to think of a relationship I detest more than my relationship with my ISP/MSO.
Yes, indeed, good for them. The ISPs are counting on general ignorance and a PR battle to win them the right to charge incoming tolls on their networks. This has to be resisted. However, I read this morning that Netflix caved and agreed to remove the messaging early. I hope it's not true, and that they keep up the fight. Three Netflix accounts in our house.
How about a link to the document instead of some fly-by-night web host with a cheesy flash PDF viewer?
Scribd isn't exactly a "fly-by-night web host". They've been around for almost 10 years already.
That is exactly the problem with Scribd? No growth no pivots, so all that is left do to is plaster the thing with ads and have someone keep it running at night.
It's terrible exactly because it is too old now.
It's terrible exactly because it is too old now.
But it's a ycombinator company, so it doesn't matter. People will blindly support it due to the nature.
I had a pretty easy time reading it. Was it difficult for you?
There's a large orange "Download" button directly to the right of the document.
Which forces you to login and then pay. Just link to the damn PDFs. Scribd is a parasite.
Scribd is a Ycombinator company. All pdf links are converted to Scribd links.
Not quite, all PDF links posted have a [Scribd] tag appended to their titles that links to a scribd mirror. However in this case the original PDF is hosted on scribd, the OP got the link from an Arstechnica article.
Usually I can click the left part of the URL for the original PDF, in this case it is just a scribd URL and when I go to it the contents is blank in this browser sadly and I can't read it. My hunch is that if a PDF URL is submitted to HN it automatically adds a scribd link to the right if it can leaving the original to the left, which is snazzy and nice for those that use scribd, but that in this case it was the scribd URL itself that was submitted.
I have just successfully created an account with a bogus email address and was then able to download the PDF without payment. Still Scribd is a PITA, especially since Firefox and Chrome got integrated PDF readers.
Please upload it somewhere for us, for example at archive.org
Here's an archive.org link: https://archive.org/details/netflix-response-to-verizon-dema...
Direct link (not sure if this is the best direct link you can get from them): https://ia902509.us.archive.org/21/items/netflix-response-to...
Direct link (not sure if this is the best direct link you can get from them): https://ia902509.us.archive.org/21/items/netflix-response-to...
They are saying "The Verizon network is crowded right now" but in the letter they say that the problem is with "interconnection congestion". So yes the problem lies with Verizon and its hesitancy to open up peering points, but if I were a Netflix lawyer that wording would have me concerned since Verizon's network is not crowded. It's like having a crowd outside a bar waiting to get in.
It's like having a crowd inside a bar waiting to get out.
It's like having a crowd inside your bar begging you for beer, then serving them in happy hour through only one tap.
Unless the guys with the beer truck out back giving you free beer to serve your paying customers, in turn pay you to open up another tap.
Unless the guys with the beer truck out back giving you free beer to serve your paying customers, in turn pay you to open up another tap.
i'm sorry, this is all going over my head... can i get this in libraries of congress over football fields?
Its like a crowd inside the library trying to get out to the football field?
The congested interconnection point is part of Verizon's network, surely.
Or is the interconnection part of Netflix's network?
Neither story really brings the entire truth to the table. Netflix could say "our connection to Verizon is full right now" which is more accurate but lacks the oomph that this is all 100% Verizon's fault.
Netflix could say "our pipes have a lot more room but Verizon will only accept so much" which I think also is perfectly accurate yet still paints Verizon as the ones who are lacking in capacity.
Neither story really brings the entire truth to the table. Netflix could say "our connection to Verizon is full right now" which is more accurate but lacks the oomph that this is all 100% Verizon's fault.
Netflix could say "our pipes have a lot more room but Verizon will only accept so much" which I think also is perfectly accurate yet still paints Verizon as the ones who are lacking in capacity.
You might like to read Level3's take on it.
A port that is on average utilised at 90 percent will be saturated, dropping packets, for several hours a day. We have congested ports saturated to those levels with 12 of our 51 peers. Six of those 12 have a single congested port, and we are both (Level 3 and our peer) in the process of making upgrades – this is business as usual and happens occasionally as traffic swings around the Internet as customers change providers.
That leaves the remaining six peers with congestion on almost all of the interconnect ports between us. Congestion that is permanent, has been in place for well over a year and where our peer refuses to augment capacity. They are deliberately harming the service they deliver to their paying customers. They are not allowing us to fulfil the requests their customers make for content.
Five of those congested peers are in the United States and one is in Europe. There are none in any other part of the world. All six are large Broadband consumer networks with a dominant or exclusive market share in their local market. In countries or markets where consumers have multiple Broadband choices (like the UK) there are no congested peers.
http://blog.level3.com/global-connectivity/observations-inte...
A port that is on average utilised at 90 percent will be saturated, dropping packets, for several hours a day. We have congested ports saturated to those levels with 12 of our 51 peers. Six of those 12 have a single congested port, and we are both (Level 3 and our peer) in the process of making upgrades – this is business as usual and happens occasionally as traffic swings around the Internet as customers change providers.
That leaves the remaining six peers with congestion on almost all of the interconnect ports between us. Congestion that is permanent, has been in place for well over a year and where our peer refuses to augment capacity. They are deliberately harming the service they deliver to their paying customers. They are not allowing us to fulfil the requests their customers make for content.
Five of those congested peers are in the United States and one is in Europe. There are none in any other part of the world. All six are large Broadband consumer networks with a dominant or exclusive market share in their local market. In countries or markets where consumers have multiple Broadband choices (like the UK) there are no congested peers.
http://blog.level3.com/global-connectivity/observations-inte...
I've read that. It was what I was thinking of when I wrote both the quotes in my comment. Netflix wants to get data to Verizon but the bottleneck is at the connection between them.
I've read that. It was what I was thinking of when I wrote both the quotes in my comment.
How remarkable.
What I got from that piece is that - if Verizon are one of the companies mentioned - Verizon customers are requesting data from Netflix.
Netflix have bought connectivity from Level3.
Level3 have peering arrangements with Verizon for sockets that are dropping packets and over which Verizon is refusing to take part in a joint connection upgrade.
Continually upgrading your connectivity in peering arrangements to cope with rising demand is, for nearly every other ISP in the world (outside certain monopoly markets in the US), completely standard practice as otherwise you die.
And Verizon is refusing to do this unless Level3's own customers pay it money.
Given this is a peering arrangement, now imagine it the other way around.
Say Level3 had it's own TV business and was asking Verizon's broadband customers to cough up money to them on top of Netflix and Verizon, if they wanted Level3 to upgrade the network that Netflix are already paying for, for a usable connection to Netflix...
Think about that for a bit, and then try and tell me that the whole affair doesn't look utterly hatstand.
How remarkable.
What I got from that piece is that - if Verizon are one of the companies mentioned - Verizon customers are requesting data from Netflix.
Netflix have bought connectivity from Level3.
Level3 have peering arrangements with Verizon for sockets that are dropping packets and over which Verizon is refusing to take part in a joint connection upgrade.
Continually upgrading your connectivity in peering arrangements to cope with rising demand is, for nearly every other ISP in the world (outside certain monopoly markets in the US), completely standard practice as otherwise you die.
And Verizon is refusing to do this unless Level3's own customers pay it money.
Given this is a peering arrangement, now imagine it the other way around.
Say Level3 had it's own TV business and was asking Verizon's broadband customers to cough up money to them on top of Netflix and Verizon, if they wanted Level3 to upgrade the network that Netflix are already paying for, for a usable connection to Netflix...
Think about that for a bit, and then try and tell me that the whole affair doesn't look utterly hatstand.
The only thing I would add to your argument is the part about ISP customers having paid the ISP for that bandwidth but which the ISP is refusing to deliver.
Prior to capitulation, Netflix purchased access to Verizon through multiple tier 1 providers. These providers are publicly known to have private arrangements regarding interconnections.
More than one of the backbone providers in question have publicly stated that Verizon/Comcast/TWC are no longer following the tenants of their private agreements.
The bare contest here is that the established order of hierarchical internet traffic is being challenged. The reason, IMO, people on HN dislike the ground shift is that it:
a) creates communication failures where there were previously fewer
b) is based on regulatory capture and monopolistic business practices
c) is not a technology driven change
d) increases the moat for all established internet services
More than one of the backbone providers in question have publicly stated that Verizon/Comcast/TWC are no longer following the tenants of their private agreements.
The bare contest here is that the established order of hierarchical internet traffic is being challenged. The reason, IMO, people on HN dislike the ground shift is that it:
a) creates communication failures where there were previously fewer
b) is based on regulatory capture and monopolistic business practices
c) is not a technology driven change
d) increases the moat for all established internet services
Interconnection points are by definition part of two networks. The question is whether the congested interconnect involves Verizon at all. I would hope that, before writing this letter, Mr. Hyman would have done his research and verified that it is indeed an interconnect that Verizon is involved with. However, it is an unanswered question.
I'm sure Netflix is carefully choosing every word, well beyond one person's responsibility. This is fire they are playing with, and they know it. They're probably treating this like a legal brief that will end up in court before a judge, because the odds of that actually happening are pretty good.
It is not an unanswered question, it is merely a question you personally have not bothered researching properly. Network engineers have thoroughly answered this question.
>>> Mr. Hyman would have done his research and verified that it is indeed an interconnect that Verizon is involved with
Is it even possible for NetFlix to know this?
Is it even possible for NetFlix to know this?
Heck, even I write mails to my ISP and tell them which interconnect points they're having issues at which a simple traceroute/mtr show me. And I mostly get positive feedback.
With a bit more sophisticated tools, you'll get more confidence. Or if you're Netflix, I'd assume you can just call up level3, ask them to resolve the transit issues you're having with Verizon, and get feedback and reports about it being verizon not upgrading their interconnect to level3. I'd not be terribly surprised if verizon is one of the 6 peers mentioned at http://blog.level3.com/global-connectivity/observations-inte...
With a bit more sophisticated tools, you'll get more confidence. Or if you're Netflix, I'd assume you can just call up level3, ask them to resolve the transit issues you're having with Verizon, and get feedback and reports about it being verizon not upgrading their interconnect to level3. I'd not be terribly surprised if verizon is one of the 6 peers mentioned at http://blog.level3.com/global-connectivity/observations-inte...
Traceroutes from a single source are poor tools for positively determining a source of latency. A traceroute only tells a unidirectional story, and an incomplete one at that.
In reality in is essential to use multiple provider/observer based tools to determine point(s) of congestion. However, it is entirely possible to empirically determine who the congested peers are between, which, and when specific geographical connections are overloaded.
In reality in is essential to use multiple provider/observer based tools to determine point(s) of congestion. However, it is entirely possible to empirically determine who the congested peers are between, which, and when specific geographical connections are overloaded.
Yes.
[deleted]
Netflix should beg Verizon to sue them for false advertising. You can imagine the PR coup when Netflix comes out victorious.
Isn't that essentially saying there's a crowd outside the bar waiting to get in because the bar itself is already packed?
No, the bar isn't packed at all. The bouncer is being a dick and only letting in the people who bribe him.
There is also a group of builders who have been waiting over a year to put up a large extension on the giant unused wasteland out back, however the bar owner has been telling them not to bother. Oh, and the bar owner also has a load of liquor stores and only bought the bar in the first place to stop the extension getting built.
Kudos to Netflix, this is great strategy. But I wish the letter were a touch better written. They might attach a written-for-laymen description of the interconnection problem (so they can keep the response itself punchy).
As I understand the problem, the congestion builds at the interconnection because Verizon's routers at that point at working at capacity. Verizon could solve the problem by adding routers at that point. I honestly don't understand what Verizon wants Netflix to do, unless it is to extend the Netflix data provision deeper into the Verizon network -- essentially adding routers at the point where Verizon is supposed to be maintaining routers. (I wonder if that is what this Open Connect program is about?)
If that's right, I wish Netflix would provide some further detail explaining the problem. Otherwise they risk having readers -- and politicians -- go into glazed eyes and presume that this is just some inscrutable battle among corporate giants.
As I understand the problem, the congestion builds at the interconnection because Verizon's routers at that point at working at capacity. Verizon could solve the problem by adding routers at that point. I honestly don't understand what Verizon wants Netflix to do, unless it is to extend the Netflix data provision deeper into the Verizon network -- essentially adding routers at the point where Verizon is supposed to be maintaining routers. (I wonder if that is what this Open Connect program is about?)
If that's right, I wish Netflix would provide some further detail explaining the problem. Otherwise they risk having readers -- and politicians -- go into glazed eyes and presume that this is just some inscrutable battle among corporate giants.
"I honestly don't understand what Verizon wants Netflix to do...."
Die in a ditch.
Seriously. Per the last big discussion on this, from https://news.ycombinator.com/item?id=7858919 from a Verizon press release, "Verizon had a total of 5.8 million FiOS Internet and 5.0 million FiOS Video connections at the end of the [2nd] quarter [of 2013]". I would infer from that press release that they have 3.1 million plain DSL customers.
Netflix et. al. are an existential threat to their FiOS Video business, especially since to my knowledge Verizon is the only holdout in putting caps on landline Internet connections (e.g. AT&T is in much less danger with 150-250 GiB/month, $10/for each additional 50 GiB).
So I'm assuming there's nothing Netflix can do to satisfy Verizon besides giving up, going out of business, etc. Which, with cap and congestion ISP policies, is not beyond the realm of possibility; since the DVD renting business went sour, I've long thought for this reason Netflix was a high risk venture.
Die in a ditch.
Seriously. Per the last big discussion on this, from https://news.ycombinator.com/item?id=7858919 from a Verizon press release, "Verizon had a total of 5.8 million FiOS Internet and 5.0 million FiOS Video connections at the end of the [2nd] quarter [of 2013]". I would infer from that press release that they have 3.1 million plain DSL customers.
Netflix et. al. are an existential threat to their FiOS Video business, especially since to my knowledge Verizon is the only holdout in putting caps on landline Internet connections (e.g. AT&T is in much less danger with 150-250 GiB/month, $10/for each additional 50 GiB).
So I'm assuming there's nothing Netflix can do to satisfy Verizon besides giving up, going out of business, etc. Which, with cap and congestion ISP policies, is not beyond the realm of possibility; since the DVD renting business went sour, I've long thought for this reason Netflix was a high risk venture.
For your second paragraph, that is what Open Connect is about -- put netflix hardware inside the ISP's own network to reduce congestion at interconnects.
That said, it's also an inscrutable battle among corporate giants, but one that people care about because in general, most people hate their ISP but like internet companies like Netflix and so put up with things.
For up-to-date analysis on Internet issues, read up on dslreports.com as each stunt happens. It's not enough to hit mainstream news often, but if you follow it online you'll pick up on patterns. Here they cover this PDF (the last paragraph links to backstory): http://www.dslreports.com/shownews/Netflix-Wont-Back-Down-on...
That said, it's also an inscrutable battle among corporate giants, but one that people care about because in general, most people hate their ISP but like internet companies like Netflix and so put up with things.
For up-to-date analysis on Internet issues, read up on dslreports.com as each stunt happens. It's not enough to hit mainstream news often, but if you follow it online you'll pick up on patterns. Here they cover this PDF (the last paragraph links to backstory): http://www.dslreports.com/shownews/Netflix-Wont-Back-Down-on...
Netflix paid Verizon to upgrade their network, but Verizon hasn't finished the upgrades because it takes them months to order and install routers. Netflix knows that these upgrades are coming but is still publicly shaming Verizon, perhaps trying to encourage them to work faster. Presumably Verizon would prefer Netflix to be patient.
Some citations for those of you who are confused:
http://arstechnica.com/tech-policy/2014/04/netflix-and-veriz...
http://online.wsj.com/articles/netflix-brushes-off-legal-thr... (may be paywalled)
http://www.cnet.com/news/verizon-vs-netflix-whats-this-reall...
http://arstechnica.com/tech-policy/2014/04/netflix-and-veriz...
http://online.wsj.com/articles/netflix-brushes-off-legal-thr... (may be paywalled)
http://www.cnet.com/news/verizon-vs-netflix-whats-this-reall...
It is so aggravating to have Verizon pitch an upgrade to FiOS Quantum(!) every time I turn my DVR on. If my 25mbps connection can't stream a 5mbps video why would I pay extra for 50mbps?
No thank you, Verizon.
No thank you, Verizon.
I'm thinking from a Consumer's stand point right now. No matter how I see it, it looks like ISPs are at fault in this matter but I've been questioning what the outcome will be for the Consumers in the end.
If Netflix doesn't pay, wouldn't Verizon and other service providers transfer the cost to Consumers? Or will this also be not allowed for ISP to do? If ISPs want to make more money and charge more, why would they not transfer the cost to Consumers if they cannot charge Netflix and the likes.
If Netflix doesn't pay, wouldn't Verizon and other service providers transfer the cost to Consumers? Or will this also be not allowed for ISP to do? If ISPs want to make more money and charge more, why would they not transfer the cost to Consumers if they cannot charge Netflix and the likes.
You make it sound like we, the customers, aren't already paying; like upgrades to hardware would never be reflected in the prices we pay. It really doesn't matter which company actually writes the check, their costs are reflected in my bill. As well as their profits, executive bonuses...
In this case, what matters is that one company is already taking my money for a particular service, half-assing the service itself, taking those profits, and trying to blame someone else.
In this case, what matters is that one company is already taking my money for a particular service, half-assing the service itself, taking those profits, and trying to blame someone else.
That's actually my point. Since upgrades, etc... all are reflected in our bill. When ISPs cannot charge Netflix and the likes in the long run, wouldn't it be reflected on the Consumer's bill? Or will the ISP business become less profitable than expected (from the eyes of ISPs)?
I see that ISPs (companies) generally want to make more money. Attempting to charge Netflix is an attempt at that. If they are not able to charge Netflix this extra amount, wouldn't it be logical for them to charge consumers more?
As it appears in Netflix's statement, their network is congested. If ISPs want to keep their profit ratio and offer better services to their consumers. Somebody has to pay more. I doubt they would settle for anything less.
Again this is just me. I would be interested to hear if there would be anything preventing ISPs from charging their customers more...
I see that ISPs (companies) generally want to make more money. Attempting to charge Netflix is an attempt at that. If they are not able to charge Netflix this extra amount, wouldn't it be logical for them to charge consumers more?
As it appears in Netflix's statement, their network is congested. If ISPs want to keep their profit ratio and offer better services to their consumers. Somebody has to pay more. I doubt they would settle for anything less.
Again this is just me. I would be interested to hear if there would be anything preventing ISPs from charging their customers more...
Verizon has so much bandwidth to get in and out of their network for broader internet bound and broader internet originating traffic. Netflix has to come in the off ramp that all the torrents, hacker news, and etc. traffic has to come in.
Netflix has offered to build a new offramp for Netflix traffic that goes directly into Verizon's network. Netflix is willing to shoulder ALL of the burden of this infrastructure cost, and to maintain it.
Verizon has refused the offer, and instead wants to charge Netflix a toll so that their traffic gets to use an EZ-Pass lane, instead of going through the normal slow toll booths that everyone else's traffic goes through.
Netflix has offered to build a new offramp for Netflix traffic that goes directly into Verizon's network. Netflix is willing to shoulder ALL of the burden of this infrastructure cost, and to maintain it.
Verizon has refused the offer, and instead wants to charge Netflix a toll so that their traffic gets to use an EZ-Pass lane, instead of going through the normal slow toll booths that everyone else's traffic goes through.
>I would be interested to hear if there would be anything preventing ISPs from charging their customers more...
Public backlash? Large ISPs are losing subscribers on their normally profitable video and voice packages. Most have a monopoly or duopoly on the data packages, but raising prices will bring more attention to their monopolistic ways, attention they will have to buy off with more and more bribes and lobbying which would also offset any profits.
Since they can't recover profits because of complaining customers and they can't lower profits because of complaining share holders they need to find alternative sources of revenue.
Public backlash? Large ISPs are losing subscribers on their normally profitable video and voice packages. Most have a monopoly or duopoly on the data packages, but raising prices will bring more attention to their monopolistic ways, attention they will have to buy off with more and more bribes and lobbying which would also offset any profits.
Since they can't recover profits because of complaining customers and they can't lower profits because of complaining share holders they need to find alternative sources of revenue.
Am I the only one who doesn't use streaming video and doesn't want my ISP rates to go up because a vast majority of the traffic they now have to deal with is video and their architecture needs to be built out in a way that it otherwise wouldn't?
The problem, as I see it, is that Verizon/Comcast (and other ISPs) are overselling, and the amount of traffic that Netflix is generating is showing that. Verizon sells X customers Y-bandwidth connections, but they don't have X*Y worth of uplinks, so they can't supply data to the people who want it.
Until now, this hasn't been a problem, because most people didn't max out their connections, and if they did they didn't do it for very long. The very nature of Netflix's business, though, is to serve the best traffic that the customer's connection can handle, and that starts to add up. Unfortunately, it adds up so much that in some cases (e.g. Comcast) you can't even serve VHS-quality content to all the customers who want it.
Because Verizon is overselling their bandwidth, they're now having issues where they can't service the demand that they sold their customers on. Instead of upgrading their upstream connections (which would cost them money, which they have a lot of), they want Netflix to pay for it. The logic is that Verizon wouldn't have these problems if Netflix weren't sending so much content; that argument breaks down when you consider that Verizon sold their services based on available bandwidth and now can't provide it; Netflix is just the service that exposed the problem.
Netflix has other solutions, though; they can provide servers that get hosted in the ISP's network to provide content to users without traffic coming from outside the ISPs' networks (thus freeing up their normal uplinks for regular traffic), or by peering with Netflix at various points (which accomplishes the same thing).
Unfortunately, neither of these solutions involves Netflix giving money to Verizon, which is why Verizon doesn't want to bother with going to the time and trouble when they can force Netflix to pay for their infrastructure upgrades instead.
Until now, this hasn't been a problem, because most people didn't max out their connections, and if they did they didn't do it for very long. The very nature of Netflix's business, though, is to serve the best traffic that the customer's connection can handle, and that starts to add up. Unfortunately, it adds up so much that in some cases (e.g. Comcast) you can't even serve VHS-quality content to all the customers who want it.
Because Verizon is overselling their bandwidth, they're now having issues where they can't service the demand that they sold their customers on. Instead of upgrading their upstream connections (which would cost them money, which they have a lot of), they want Netflix to pay for it. The logic is that Verizon wouldn't have these problems if Netflix weren't sending so much content; that argument breaks down when you consider that Verizon sold their services based on available bandwidth and now can't provide it; Netflix is just the service that exposed the problem.
Netflix has other solutions, though; they can provide servers that get hosted in the ISP's network to provide content to users without traffic coming from outside the ISPs' networks (thus freeing up their normal uplinks for regular traffic), or by peering with Netflix at various points (which accomplishes the same thing).
Unfortunately, neither of these solutions involves Netflix giving money to Verizon, which is why Verizon doesn't want to bother with going to the time and trouble when they can force Netflix to pay for their infrastructure upgrades instead.
All ISPs oversell; the two alternatives are (a) really low bandwidth for everyone, or (b) really expensive bandwidth with lots of wasted capacity that people seldom use. If you're arguing against contention, you're arguing for a massively inefficient allocation of money. This is basically a thin provisioning vs thick provisioning argument. Thin provisioning makes lots of sense when not everyone is using the maximum all the time, and especially when thick provisioning is expensive. Uncontended bandwidth is expensive.
If customers who watch Netflix are using more bandwidth on average than customers who don't watch Netflix, who should pay more?
If you think Netflix shouldn't pay more for their bandwidth (and then take it out of their customer), you're arguing that Verizon should pay for the extra bandwidth (and thus take it out of their customer).
The logical conclusion is that Netflix-watching users would be subsidized by non-Netflix-watching users.
Alternatives might be a special "Netflix" package sold by Verizon that has better quality interconnects to Netflix. Would you agree with that? Not very net-neutral though.
If customers who watch Netflix are using more bandwidth on average than customers who don't watch Netflix, who should pay more?
If you think Netflix shouldn't pay more for their bandwidth (and then take it out of their customer), you're arguing that Verizon should pay for the extra bandwidth (and thus take it out of their customer).
The logical conclusion is that Netflix-watching users would be subsidized by non-Netflix-watching users.
Alternatives might be a special "Netflix" package sold by Verizon that has better quality interconnects to Netflix. Would you agree with that? Not very net-neutral though.
The statement that all ISP's over-sell is oversimplified.
All ISPs statistically multiplex the bandwidth requirements for off-net connectivity. Over-selling a peering relationship is impossible, as the ISP is selling only a speed-rated connection to it's own network. The assumption (or contractual service level) is generally that the ISP will augment either transit or peering as the resource becomes saturated.
The issue on hand is not the local or last mile bandwidth between the (retail) ISP and the customer, but the interconnection between the retail and the wholesale bandwidth providers. Two of the wholesale providers in question have publicly stated that their interconnections with specific retail providers are congested, and are not being augmented per their private contracts with the retail networks.
The above explanation is also a simplification, because for example Verizon is also a premium wholesale network in North America and several MSOs have also been building national transport networks.
All ISPs statistically multiplex the bandwidth requirements for off-net connectivity. Over-selling a peering relationship is impossible, as the ISP is selling only a speed-rated connection to it's own network. The assumption (or contractual service level) is generally that the ISP will augment either transit or peering as the resource becomes saturated.
The issue on hand is not the local or last mile bandwidth between the (retail) ISP and the customer, but the interconnection between the retail and the wholesale bandwidth providers. Two of the wholesale providers in question have publicly stated that their interconnections with specific retail providers are congested, and are not being augmented per their private contracts with the retail networks.
The above explanation is also a simplification, because for example Verizon is also a premium wholesale network in North America and several MSOs have also been building national transport networks.
The logical conclusion is that Netflix-watching users would be subsidized by non-Netflix-watching users.
Verizon chose that business model; we're just asking them to stick to it. Or they could meter. But charging both sides is bad for innovation.
Verizon chose that business model; we're just asking them to stick to it. Or they could meter. But charging both sides is bad for innovation.
If we're going with metering, I hope cable television watchers can join in the fun. If I subscribe for cable television and I only watch two hours a month, why should I subsidize someone who uses it 12 hours a day?
The argument based on costs is that it costs a cableco, or e.g. satellite company, no additional incremental cost to serve you and the 12 hours a day customer, all bits for all channels are broadcast to all customers.
Whereas each additional outside their net a la carte video stream does cost, if it's at a period of peak demand. The build out for peak demand is what costs, well, everyone, including the last mile ISPs, and video does tend to coincide with or now probably creates peak demand in evenings.
Some technical solutions like pulling during off peak and storing it for later viewing are unacceptable to content providers :-(, then again, none of this matters in a good way to a last mile company who very much wants you paying them for video rather than Netflix et. al., which is all of the big US ones except Century (formerly US Worst).
Interestingly, the last time I checked, a few years ago, one of CableOne's standard capping mechanisms was only for usage at peak periods....
Whereas each additional outside their net a la carte video stream does cost, if it's at a period of peak demand. The build out for peak demand is what costs, well, everyone, including the last mile ISPs, and video does tend to coincide with or now probably creates peak demand in evenings.
Some technical solutions like pulling during off peak and storing it for later viewing are unacceptable to content providers :-(, then again, none of this matters in a good way to a last mile company who very much wants you paying them for video rather than Netflix et. al., which is all of the big US ones except Century (formerly US Worst).
Interestingly, the last time I checked, a few years ago, one of CableOne's standard capping mechanisms was only for usage at peak periods....
> The argument based on costs is that it costs a cableco, or e.g. satellite company, no additional incremental cost to serve you and the 12 hours a day customer, all bits for all channels are broadcast to all customers.
Doesn't matter. The person watching 12 hours a day is getting a far greater value from their subscription than I am, which is the same reason the (generally all-encompassing, as opposed to peak metering) caps are being argued as fair, because of the value received.
Now of course all the ISPs policies make sense when you assume their motives are to kneecap a competitor and not to fairly price their service.
Doesn't matter. The person watching 12 hours a day is getting a far greater value from their subscription than I am, which is the same reason the (generally all-encompassing, as opposed to peak metering) caps are being argued as fair, because of the value received.
Now of course all the ISPs policies make sense when you assume their motives are to kneecap a competitor and not to fairly price their service.
Or maybe the opposite ?
If you charged Netflix, they would consider investing more money on trying to reduce the bandwidth they use: - innovative video codec - better encoders - new streaming technology
If you charged Netflix, they would consider investing more money on trying to reduce the bandwidth they use: - innovative video codec - better encoders - new streaming technology
(Ignoring sunk costs:)
The ongoing costs to an ISP are not the costs of providing bits. Let me hand-wave a bit and say the 99th percentile of families is using 4 hours of Netflix per day, and that's about 1 GB per hour, so 120GB per month, which is maybe a few bucks of costs to the ISP.
The ongoing costs to an ISP are constantly building out and upgrading the network so that when we want to watch Netflix on Tuesday of next year at the peak hour we can. If there is metering or preference used, it should only apply at the peak times.
The ongoing costs to an ISP are not the costs of providing bits. Let me hand-wave a bit and say the 99th percentile of families is using 4 hours of Netflix per day, and that's about 1 GB per hour, so 120GB per month, which is maybe a few bucks of costs to the ISP.
The ongoing costs to an ISP are constantly building out and upgrading the network so that when we want to watch Netflix on Tuesday of next year at the peak hour we can. If there is metering or preference used, it should only apply at the peak times.
The problem isn't that they're overselling per se, the problem is the deception involved. They sill this "up to x megs" service, with the knowledge of that overselling in mind, they can't ever provide that speed. In effect, selling you "up to" some amount of bandwidth when it is mathematically impossible for you to get that amount, ever.
False advertising.
In a perfect world: I'd like to see a "broadband facts" that ISPs are required to provide, including average single-subscriber connection throughput to the their subscriber's top 100 sites, equivalent packet loss and jitter, amount of oversell (combined max subscriber rates vs uplink/downlink capacity). This data must be posted, prominently, on all sales literature and sites, and communicated verbally over phone. It must be updated no less than once per month.
False advertising.
In a perfect world: I'd like to see a "broadband facts" that ISPs are required to provide, including average single-subscriber connection throughput to the their subscriber's top 100 sites, equivalent packet loss and jitter, amount of oversell (combined max subscriber rates vs uplink/downlink capacity). This data must be posted, prominently, on all sales literature and sites, and communicated verbally over phone. It must be updated no less than once per month.
Verizon doesn't have to raise prices; they made $5.1bn in profit last year: http://money.cnn.com/2014/01/21/technology/mobile/verizon-ea...
They just don't want to have to re-invest it in their network if they can force Netflix to do so instead.
They just don't want to have to re-invest it in their network if they can force Netflix to do so instead.
An absolute figure tells us nothing. Unless any company that has a profit should never raise prices on anything.
And just to show how the numbers can be cherry-picked, in 2012 Verizon had a net profit margin of less than 1 percent.
And just to show how the numbers can be cherry-picked, in 2012 Verizon had a net profit margin of less than 1 percent.
I appreciate your POV that what you have now is "good enough" and it doesn't feel right to be taken along to support other people's habits. (And it's totally rotten you got downvoted.)
However, ISPs are a shared resource. Just like I tell people who act shocked when their speeds aren't guaranteed, sometimes the needs of the shared network override yours. You are getting a tremendous discount over a private line by bundling your whims with a bunch of your neighbors.
That said, if your current service is "good enough," you might try using wireless or DSL.
However, ISPs are a shared resource. Just like I tell people who act shocked when their speeds aren't guaranteed, sometimes the needs of the shared network override yours. You are getting a tremendous discount over a private line by bundling your whims with a bunch of your neighbors.
That said, if your current service is "good enough," you might try using wireless or DSL.
Probably you aren't the only one, but then again I don't, either.
It is a possibility (likelihood, even) that Verizon and other providers will raise rates, even though they aren't delivering on their commitments at current rates. I frankly consider residential ISP service providers who do what Verizon does to be in breach of the spirit of their contracts, even if the "best effort" language offers weasel-word loopholes out of being in breach (legal) of their contracts. I don't think they should be allowed to raise their rates until they demonstrate they can meet their current commitments under their current rate structures. But there is an infinitesimally small chance of regulatory, legislative, or court actions that will enforce that, especially in this country today, so I'm basically just waiting for the boost in monthly cost for my internet service.
It is a possibility (likelihood, even) that Verizon and other providers will raise rates, even though they aren't delivering on their commitments at current rates. I frankly consider residential ISP service providers who do what Verizon does to be in breach of the spirit of their contracts, even if the "best effort" language offers weasel-word loopholes out of being in breach (legal) of their contracts. I don't think they should be allowed to raise their rates until they demonstrate they can meet their current commitments under their current rate structures. But there is an infinitesimally small chance of regulatory, legislative, or court actions that will enforce that, especially in this country today, so I'm basically just waiting for the boost in monthly cost for my internet service.
Yes, you are the only one, or close enough that it makes little difference.
Back in Ye Olden Dayes, when techies were the main audience for technical products, we could reasonably expect offerings that catered to our needs. But for quite some time, the industry has been driven by mainstream consumers. E.g., the last smartphone built for a nerd audience was the Palm Treo. The current ones are all consumer focused.
We're along for the ride now, and video in particular has been a major driver for bandwidth demand.
It could be that there will be enough we-don't-need-much-bandwidth consumers that somebody will offer a not-good-enough-to-stream package that will be a better deal for you. But again, that's a consumer pricing option.
Back in Ye Olden Dayes, when techies were the main audience for technical products, we could reasonably expect offerings that catered to our needs. But for quite some time, the industry has been driven by mainstream consumers. E.g., the last smartphone built for a nerd audience was the Palm Treo. The current ones are all consumer focused.
We're along for the ride now, and video in particular has been a major driver for bandwidth demand.
It could be that there will be enough we-don't-need-much-bandwidth consumers that somebody will offer a not-good-enough-to-stream package that will be a better deal for you. But again, that's a consumer pricing option.
When video is consuming most of the internet transport links, it only makes sense that website operators should also shoulder rate hikes. /s
We who understand how the network works and what is actually happening to it and simultaneously observing the net-neutrality screaming of most people should take note. People do not behave rationally, nor do they care about anything nearly as much as their internet cat videos or "house of cards" episodes.
We who understand how the network works and what is actually happening to it and simultaneously observing the net-neutrality screaming of most people should take note. People do not behave rationally, nor do they care about anything nearly as much as their internet cat videos or "house of cards" episodes.
You are probably one of the few. But I wouldn't buy too much into the argument that their rates have to go up. Internet speeds are higher and congestion are lower in many other countries where their rates are cheaper. I think there is one reason and one reason alone for this. That reason is localized monopolies. Take a quick gander into the deals the ISPs cut with local government. Prepare to be disturbed.
I think there is one reason and one reason alone for this.
If there's "one reason," it's because the US is spread out. Of the countries with faster consumer Internet speeds than the US, the biggest in area is France, which is less than a tenth the size of the lower 48.
That said, local governments often suck. A few months ago Google just gave up on putting Google Fiber into San Francisco.
Take a quick gander into the deals the ISPs cut with local government. Prepare to be disturbed.
Could you show us the example you have in mind?
If there's "one reason," it's because the US is spread out. Of the countries with faster consumer Internet speeds than the US, the biggest in area is France, which is less than a tenth the size of the lower 48.
That said, local governments often suck. A few months ago Google just gave up on putting Google Fiber into San Francisco.
Take a quick gander into the deals the ISPs cut with local government. Prepare to be disturbed.
Could you show us the example you have in mind?
The limiting factor is not land area, but the marginal cost to serve an additional subscriber. It isn't the size of your graph, but the total cost of your spanning tree. The numbers involved are linear distances and number of nodes, not areas.
In most areas of the US, pre-existing infrastructure, such as improved roads and utility easements, makes serving additional customers relatively low cost. But monopolies and commodity suppliers operate at different supply points. A monopoly will intentionally reduce output below the point where marginal cost equals marginal revenue, to achieve higher prices and economic profits.
Leaving aside the concept of natural monopoly, that's the one reason. Most telecom markets are a local monopoly. Service sucks because the company providing it makes more money that way.
Land area and population density are red herrings. You need to measure the size of existing networks, such as roads, electric power, potable water and sewers, and divide those by the number of people served.
To use a car analogy, think about the Autobahn-style Interstate highway system. Before and after it was constructed, places remain the same absolute distance apart. But afterward, traveling between those places could take more or less time. Places that were previously adjacent might now require a detour via an overpass, whereas places previously distant might both have convenient on and off ramps. Travel times by car are thus determined by the roads network topology and not purely geographical distribution.
In most areas of the US, pre-existing infrastructure, such as improved roads and utility easements, makes serving additional customers relatively low cost. But monopolies and commodity suppliers operate at different supply points. A monopoly will intentionally reduce output below the point where marginal cost equals marginal revenue, to achieve higher prices and economic profits.
Leaving aside the concept of natural monopoly, that's the one reason. Most telecom markets are a local monopoly. Service sucks because the company providing it makes more money that way.
Land area and population density are red herrings. You need to measure the size of existing networks, such as roads, electric power, potable water and sewers, and divide those by the number of people served.
To use a car analogy, think about the Autobahn-style Interstate highway system. Before and after it was constructed, places remain the same absolute distance apart. But afterward, traveling between those places could take more or less time. Places that were previously adjacent might now require a detour via an overpass, whereas places previously distant might both have convenient on and off ramps. Travel times by car are thus determined by the roads network topology and not purely geographical distribution.
Land area and population density are red herrings
No, when trying to wire up a population, population density really really matters. The #1 and #2 countries for Internet speed are Singapore and Hong Kong.
No, when trying to wire up a population, population density really really matters. The #1 and #2 countries for Internet speed are Singapore and Hong Kong.
Only in the sense that high population density makes minimal spanning trees very small indeed. The area simply is not relevant for wired coverage. Wireless links are another story, obviously, where antennas define a coverage area, but as long as the bits go through linear fibers, it simply does not matter how large someone's back yard is, or how far it extends from the front door. That area is irrelevant to the provision of service (unless someone lives on the other side of it).
The fastest data networks are wired, and even the wireless networks have linear backhaul.
By topology, a high-rise apartment building where everyone is within 100m of the utility closet on their floor is not all that different from a small town where most houses are within 100m of Main Street. The apartments have smaller area because people are stacked on top of each other. The total length of the cables and the equipment at the distribution nodes are still what matters.
You are removing a step in the causality chain. High population density causes efficient networks because all high-density areas incorporate their vertical space, by necessity. High-rise apartment and office buildings make it relatively easy to wire up a lot of people all at once.
But low population density does not necessarily imply a costly, inefficient network. The correlation between the two is stronger at the dense end of the scale. In the case where information about network topology is not available, population density may be used as a less accurate substitute, but your conclusions will likewise be less accurate, especially at the lower end of the scale.
Probably a closer approximation could be reached by looking at aerial photos of the places under comparison, adding up the total length of visible streets, and dividing population totals by that number, to get people per street-meter rather than people per square-meter.
The fastest data networks are wired, and even the wireless networks have linear backhaul.
By topology, a high-rise apartment building where everyone is within 100m of the utility closet on their floor is not all that different from a small town where most houses are within 100m of Main Street. The apartments have smaller area because people are stacked on top of each other. The total length of the cables and the equipment at the distribution nodes are still what matters.
You are removing a step in the causality chain. High population density causes efficient networks because all high-density areas incorporate their vertical space, by necessity. High-rise apartment and office buildings make it relatively easy to wire up a lot of people all at once.
But low population density does not necessarily imply a costly, inefficient network. The correlation between the two is stronger at the dense end of the scale. In the case where information about network topology is not available, population density may be used as a less accurate substitute, but your conclusions will likewise be less accurate, especially at the lower end of the scale.
Probably a closer approximation could be reached by looking at aerial photos of the places under comparison, adding up the total length of visible streets, and dividing population totals by that number, to get people per street-meter rather than people per square-meter.
> If there's "one reason," it's because the US is spread out.
Stop trotting this meme out. It's false.
South Korea invested 1.08 billion over about six years, from 1999 to 2005. They also deregulated, primarily around competition - direct competition is allowed between ISPs there (it is not here, usually due to locally determined monopoly status).
South Korea has a landmass of approximately 100,000 square kilometers. Which calculates out to about 10,000 invested per square kilometer.
From the mid 90s to the mid 00s, internet service providers received a sum of over 200 billion USD (some say as high as 300 billion USD) in direct and tax subsidies, with the understanding that they would build out fiber to the home. It never happened, for various reasons. But the point is we already tried subsidies to get it, and it didn't work.
According to the 2010 census, there are 486 urbanized areas and 3087 urban clusters. UAs are 50,000 or more people, UCs are at least 2,500 and less than 50,000 people. Pretty much covers everything from small towns in the middle of nowhere to large metropolitan areas like NYC but excludes Yosemite, most of Alaska, etc -- you know, the places where almost no one lives and probably don't even have cell reception. UAs and UCs combined, according to US Census data from 2010, cover 1,565,052.983 km^2. If we pretend that ISPs weren't planning on wiring up rural areas with fiber anyways, that means we spent $127,791 per square kilometer and got nothing for it. (For the record, the average population density of UAs and UCs together is 978.54/km^2). This would cover 80.7% of the total US population.
Per square kilometer of populated area, we spent over twelve times what South Korea did to get fiber to the home and got, basically, nothing for it.
Stop trotting this meme out. It's false.
South Korea invested 1.08 billion over about six years, from 1999 to 2005. They also deregulated, primarily around competition - direct competition is allowed between ISPs there (it is not here, usually due to locally determined monopoly status).
South Korea has a landmass of approximately 100,000 square kilometers. Which calculates out to about 10,000 invested per square kilometer.
From the mid 90s to the mid 00s, internet service providers received a sum of over 200 billion USD (some say as high as 300 billion USD) in direct and tax subsidies, with the understanding that they would build out fiber to the home. It never happened, for various reasons. But the point is we already tried subsidies to get it, and it didn't work.
According to the 2010 census, there are 486 urbanized areas and 3087 urban clusters. UAs are 50,000 or more people, UCs are at least 2,500 and less than 50,000 people. Pretty much covers everything from small towns in the middle of nowhere to large metropolitan areas like NYC but excludes Yosemite, most of Alaska, etc -- you know, the places where almost no one lives and probably don't even have cell reception. UAs and UCs combined, according to US Census data from 2010, cover 1,565,052.983 km^2. If we pretend that ISPs weren't planning on wiring up rural areas with fiber anyways, that means we spent $127,791 per square kilometer and got nothing for it. (For the record, the average population density of UAs and UCs together is 978.54/km^2). This would cover 80.7% of the total US population.
Per square kilometer of populated area, we spent over twelve times what South Korea did to get fiber to the home and got, basically, nothing for it.
You should be a proponent of metered billing then. You'd pay for what you use.
Would you be less concerned if the traffic were bittorrent, or itunes, or $somethingElse?
You're absolutely correct, IMO.
This argument is over whether or not non-Netflix users will end up subsidizing Netflix watchers.
The bandwidth needs to be paid for, at the end of the day. The only thing anybody is arguing about is how the balance of payments work out.
This argument is over whether or not non-Netflix users will end up subsidizing Netflix watchers.
The bandwidth needs to be paid for, at the end of the day. The only thing anybody is arguing about is how the balance of payments work out.
I don't know the right answer, but I think the issue is more complex than the typical Netflix-good/ISP-bad dogma which is usually all I see in response to it and seems to be driven more by consumer sentiment for the respective companies than anything else.
If ISPs can't afford to upgrade their facilities to handle streaming video at the current rates, why do you assume ISPs would just raise the price of all plans across the board rather than create "streaming video" plans that cost more?
Maybe... I grew up in Tucson, and and about 40 years ago they wanted to put in some cross town freeways. A bunch of transplants complained "ohh we don't want to be like LA" and so the freeways weren't built... Did that stop people from moving to Tucson and cluttering up the roads there?
More people want to stream video. More people are using the internet on more devices (AND the ISPs are encouraging it!)
Doesn't matter if it is Netflix or not.
You probably are, for myself most of my internet traffic comes from streaming. Twitch/YouTube/Amazon etc. I rarely watch normal TV nowadays and i dont want to be bound to someone who decides when and what i have to watch.
And i expect that my ISP deliverers the internet speed i pay for and upgrade his infrastructure accordingly without raising prices.
And i expect that my ISP deliverers the internet speed i pay for and upgrade his infrastructure accordingly without raising prices.
Your advocacy in support of ISP monopolies will only end up costing you more should you be successful. What you pay your ISP has little to do with their infrastructure. Look at other countries with similar subscribers/km2. Rates you're paying are just profit (and lobbyists) from lack of competition.
Is that real? It doesn't read like it was written by an attorney, and there was even a grammar mistake in the first paragraph.
I had my doubts, too: it seemed like a very strange way to "leak" the response, and the wording and phrasing weren't perfect. ARS seems to have called a spokesperson who didn't deny it, though:
http://arstechnica.com/tech-policy/2014/06/netflix-refuses-t...
Or at least ARS has a spokesperson saying the "letter speaks for itself," which suggests authenticity.
http://arstechnica.com/tech-policy/2014/06/netflix-refuses-t...
Or at least ARS has a spokesperson saying the "letter speaks for itself," which suggests authenticity.
Real attorneys make grammar mistakes all the time.
You're telling me that no one proofreads documents like these?
hopefully netflix continues this push toward broadband transparency. as justice louis brandeis said, "sunlight is said to be the best of disinfectants." it seems unreasonable for verizon and broadband providers to advertise, and charge consumers for, high-quality networks -- then try to pry money from data suppliers like netflix when usage rises. that said, is there a fair counterargument that justifies verizon's desire to charge both consumers for high-speed access and netflix for increased network usage?
> is there a fair counterargument that justifies verizon's desire to charge both consumers for high-speed access and netflix for increased network usage?
The closest approximation is "as a publicly traded company they owe it to their shareholders to make as much money as possible"
The closest approximation is "as a publicly traded company they owe it to their shareholders to make as much money as possible"
What's the Open Connect program they mention? how does it work?
ISPs can directly connect their networks to Open Connect for free. ISPs can do this either by free peering with us at common Internet exchanges, or can save even more transit costs by putting our free storage appliances in or near their network.
Major ISPs around the world have already connected to Open Connect, including Frontier, British Telecom, TDC, Clearwire, GVT, Telus, Bell Canada, Virgin, Cablevision, Google Fiber, Telmex, and more
https://www.netflix.com/openconnect
Major ISPs around the world have already connected to Open Connect, including Frontier, British Telecom, TDC, Clearwire, GVT, Telus, Bell Canada, Virgin, Cablevision, Google Fiber, Telmex, and more
https://www.netflix.com/openconnect
Cablevision's Netflix offering is in fantastic "Super HD" because Cablevision did the right thing and let Netflix store and manage content inside Cablevision's network so it doesn't flow over peering edges. The quality is shocking.
But then, Cablevision in general is shocking -- when you compare to most other cable ISPs. I can sustain 130+ megabits day or night on a 100 megabit service tier, with the reliability of dial tone.
Other ISPs need to follow the Netflix/Cablevision partnership model. People who have that, worry about moving and losing it.
But then, Cablevision in general is shocking -- when you compare to most other cable ISPs. I can sustain 130+ megabits day or night on a 100 megabit service tier, with the reliability of dial tone.
Other ISPs need to follow the Netflix/Cablevision partnership model. People who have that, worry about moving and losing it.
As far as I'm aware, not all ISPs get that deal - the ISPs need to pay Netflix for it.
There are four invariants as I see it:
1) Netflix is using a significant amount of a limited resource (bandwidth) 2) If Netflix doesn't work, consumers blame their ISP, not Netflix 3) The consumer is going to pay for this bandwidth, whether via ISP or Netflix 4) Consumers don't like caps or understand special packages that differentiate between packets routed one way or another
If Netflix doesn't pay for this bandwidth, then ISPs need to pay for it - and consumers will pay more to their ISPs to cover these costs. That means that consumers that don't use Netflix will be subsidizing consumers who do use Netflix.
IMO ISPs should charge Netflix so that Netflix can charge higher prices to users who are using more bandwidth via Netflix. That better reflects the link between who's using a limited resource and who's paying for it. Unfortunately, because of point (2) above, Netflix has quite a strong bargaining position.
There are four invariants as I see it:
1) Netflix is using a significant amount of a limited resource (bandwidth) 2) If Netflix doesn't work, consumers blame their ISP, not Netflix 3) The consumer is going to pay for this bandwidth, whether via ISP or Netflix 4) Consumers don't like caps or understand special packages that differentiate between packets routed one way or another
If Netflix doesn't pay for this bandwidth, then ISPs need to pay for it - and consumers will pay more to their ISPs to cover these costs. That means that consumers that don't use Netflix will be subsidizing consumers who do use Netflix.
IMO ISPs should charge Netflix so that Netflix can charge higher prices to users who are using more bandwidth via Netflix. That better reflects the link between who's using a limited resource and who's paying for it. Unfortunately, because of point (2) above, Netflix has quite a strong bargaining position.
Free is a misnomer and the misnomer that is driving this whole debate.
There is a cost involved with peering, even if the peering relationship is "free". That cost is the crux of much of these issues today, as Verizon (and Comcast and others) feel that Netflix should pay for their own in-house upgrades, because Netflix is creating the need for those upgrades.
Netflix argues, and I agree, that the customer is creating the demand for the data, and the customer is paying for access to the data, and the customer is paying for the infrastructure to get the data they're requesting.
OpenConnect means "You're welcome to spend a bunch of money upgrading your interconnection to our free network"
There is a cost involved with peering, even if the peering relationship is "free". That cost is the crux of much of these issues today, as Verizon (and Comcast and others) feel that Netflix should pay for their own in-house upgrades, because Netflix is creating the need for those upgrades.
Netflix argues, and I agree, that the customer is creating the demand for the data, and the customer is paying for access to the data, and the customer is paying for the infrastructure to get the data they're requesting.
OpenConnect means "You're welcome to spend a bunch of money upgrading your interconnection to our free network"
This would be true if not for the second half of the program:
>or can save even more transit costs by putting our free storage appliances in or near their network.
They don't have to interconnect to anyone. Netflix is even offering them storage devices to keep the traffic internal to their own networks.
>or can save even more transit costs by putting our free storage appliances in or near their network.
They don't have to interconnect to anyone. Netflix is even offering them storage devices to keep the traffic internal to their own networks.
> OpenConnect means "You're welcome to spend a bunch of money upgrading your interconnection to our free network"
Or colo their equipment https://www.netflix.com/openconnect/hardware . If upgrading interconnects is such an expensive issue, surely these ISPs could find 4u somewhere to stick one of these boxes in?
Or colo their equipment https://www.netflix.com/openconnect/hardware . If upgrading interconnects is such an expensive issue, surely these ISPs could find 4u somewhere to stick one of these boxes in?
Constantly upgrading telecoms infrastructure to cope with increasing demand is the only way to stay in business if you are a telecoms company, unless you have a monopoly. If they think realtime video is bad, they are really screwed in the long run.
Quasi-realtime video has the special distinction of competing with their own video offerings, e.g. see https://news.ycombinator.com/item?id=7873331 for some Verizon numbers.
Can you explain why ISPs should pay to run your servers in their network and not do the same for my personal homepage?
That's a mischaracterization of the situation.
ISPs complained about the amount of traffic originating from Netflix.
This traffic was requested by the ISP's customers, and the bandwidth to carry it was promised by the ISP.
But, still, the ISPs were unhappy. So Netflix approached them and said they are willing to offer local caches of Netflix content, for free, to any ISP that requested it to reduce network congestion.
This was a gesture of good will by Netflix, not an attempt to outsource their hosting costs. It is, fiscally, a loss to Netflix because they need to purchase and maintain the storage appliances for open connect partners.
ISPs complained about the amount of traffic originating from Netflix.
This traffic was requested by the ISP's customers, and the bandwidth to carry it was promised by the ISP.
But, still, the ISPs were unhappy. So Netflix approached them and said they are willing to offer local caches of Netflix content, for free, to any ISP that requested it to reduce network congestion.
This was a gesture of good will by Netflix, not an attempt to outsource their hosting costs. It is, fiscally, a loss to Netflix because they need to purchase and maintain the storage appliances for open connect partners.
Actually I don't think the fiscal loss is so clear-cut. Each deployment of cache nodes removes load/bandwidth from the rest of the CDN, and also reduces support time spent with customers.
In fact, 100% of Netflix traffic outside the US and 90% inside the US is served from Open Connect[0]. It is actually a shift of the location of the serving hardware that allows both parties (the ISP and Netflix) to save on bandwidth. This is a net win for everyone.
[0] http://arstechnica.com/information-technology/2014/05/netfli...
In fact, 100% of Netflix traffic outside the US and 90% inside the US is served from Open Connect[0]. It is actually a shift of the location of the serving hardware that allows both parties (the ISP and Netflix) to save on bandwidth. This is a net win for everyone.
[0] http://arstechnica.com/information-technology/2014/05/netfli...
Because your homepage doesn't attract many TB of traffic?
So, bandwidth. What is the threshold with which ISPs should start giving out free hosting?
It's not free hosting. It's not even hosting a webpage because it's not internet facing at all. Comcast customers can't visit the "free webpage" hosted on Verizon because it's not the internet we're discussing here. It's delivering a service to the local ISP network subscribers.
The ISP is allowing Netflix to bring caches and servers to their interconnect so that the ISP can meet the demands of its users who are requesting that data.
When should an ISP give out "free" things? When the quality of service for their customers requests depends on it.
It's that simple: Customers request data, and ISPs have an obligation to deliver that data once it arrives at their local network.
What you call "Free hosting" and "free upgrades", I call "upgrades for requested service paid for by my monthly bill".
Because when I request data, and that data requires infrastructure to reach me adequately, then I am paying for that infrastructure by virtue of my monthly bill, and my decision to use part of my capped bandwidth on that service.
Remember: residential internet is capped because "users must pay for their use, and infrastructure can't handle too much demand so we need to charge users to upgrade according to their usage pattern". So therefore, I'm paying for that Netflix interconnect, it's not free, I pay for it, because I pay for 300GB of data a month and all of the necessary infrastructure to deliver it!
The ISP is allowing Netflix to bring caches and servers to their interconnect so that the ISP can meet the demands of its users who are requesting that data.
When should an ISP give out "free" things? When the quality of service for their customers requests depends on it.
It's that simple: Customers request data, and ISPs have an obligation to deliver that data once it arrives at their local network.
What you call "Free hosting" and "free upgrades", I call "upgrades for requested service paid for by my monthly bill".
Because when I request data, and that data requires infrastructure to reach me adequately, then I am paying for that infrastructure by virtue of my monthly bill, and my decision to use part of my capped bandwidth on that service.
Remember: residential internet is capped because "users must pay for their use, and infrastructure can't handle too much demand so we need to charge users to upgrade according to their usage pattern". So therefore, I'm paying for that Netflix interconnect, it's not free, I pay for it, because I pay for 300GB of data a month and all of the necessary infrastructure to deliver it!
Econ 101: all things being equal cheaper is better than more expensive.
If I have a network and my customers use Netflix and aren't going to not use Netflix because I don't want them to, then part of my job is to ensure that my customers get the bandwidth I've promised them. (This isn't how it ACTUALLY works but how it's SUPPOSED to work)
As an ISP I have really rather large amounts of bandwidth in the last-mile at least in aggregate. Let's say that I can reasonably offer 20Mbps to each of my 1mm customers from my POPs to their houses. That's 20Tbps in aggregate. I probably don't have 20Tbps worth of back-haul from all my POPs to all the peering stations where I actually get the customers connected to the internet at large.
If network traffic is all long-tailed and the biggest use of bandwidth is 1% of capacity and it goes down from there "free hosting" doesn't make sense. But what if traffic to one company makes up 30% (or 80%) of total back-haul utilization at peak hours? I'm spending a lot of capacity for a single destination.
Now what if that place offered to create a magical wormhole from their servers to my customers at my POPs such that a large fraction -- say 80% -- of my customer's traffic from/to them never hits my back-haul it just appears out of thin air at the POP. Would I consider this a good deal? Depends on how much I pay for the back-haul versus how much electricity they're going to use at the POP. All-in I would suspect that it is a good deal thinking in these terms.
That's precisely what the Netflix appliance is. It's a way to give the customers Netflix without costing any bandwidth on the back-haul network that ISPs operate.
The reason that ISPs aren't all jumping right on this (despite the likely cost-savings) is that they view Netflix as the competition and they're prefer an adversarial relationship that hopefully puts Netflix out of business rather than cooperating and in their minds speeding their own demise.
So the rule of thumb you're looking for is that ISPs should start giving out free hosting when it's cheaper to give free hosting than to pay for the back-haul.
If I have a network and my customers use Netflix and aren't going to not use Netflix because I don't want them to, then part of my job is to ensure that my customers get the bandwidth I've promised them. (This isn't how it ACTUALLY works but how it's SUPPOSED to work)
As an ISP I have really rather large amounts of bandwidth in the last-mile at least in aggregate. Let's say that I can reasonably offer 20Mbps to each of my 1mm customers from my POPs to their houses. That's 20Tbps in aggregate. I probably don't have 20Tbps worth of back-haul from all my POPs to all the peering stations where I actually get the customers connected to the internet at large.
If network traffic is all long-tailed and the biggest use of bandwidth is 1% of capacity and it goes down from there "free hosting" doesn't make sense. But what if traffic to one company makes up 30% (or 80%) of total back-haul utilization at peak hours? I'm spending a lot of capacity for a single destination.
Now what if that place offered to create a magical wormhole from their servers to my customers at my POPs such that a large fraction -- say 80% -- of my customer's traffic from/to them never hits my back-haul it just appears out of thin air at the POP. Would I consider this a good deal? Depends on how much I pay for the back-haul versus how much electricity they're going to use at the POP. All-in I would suspect that it is a good deal thinking in these terms.
That's precisely what the Netflix appliance is. It's a way to give the customers Netflix without costing any bandwidth on the back-haul network that ISPs operate.
The reason that ISPs aren't all jumping right on this (despite the likely cost-savings) is that they view Netflix as the competition and they're prefer an adversarial relationship that hopefully puts Netflix out of business rather than cooperating and in their minds speeding their own demise.
So the rule of thumb you're looking for is that ISPs should start giving out free hosting when it's cheaper to give free hosting than to pay for the back-haul.
> rather than cooperating and in their minds speeding their own demise
It makes sense, then, that one of the few large cable companies that uses Open Connect is Cablevision, who's CEO is on the record saying "Ultimately over the long term I think that the whole video product is eventually going to go to the Internet."[1]
He's one of a few that has accepted the eventual fate of cable TV, and so his business decisions aren't biased by a need to delay the inevitable.
[1]: http://online.wsj.com/news/articles/SB1000142412788732342060...
It makes sense, then, that one of the few large cable companies that uses Open Connect is Cablevision, who's CEO is on the record saying "Ultimately over the long term I think that the whole video product is eventually going to go to the Internet."[1]
He's one of a few that has accepted the eventual fate of cable TV, and so his business decisions aren't biased by a need to delay the inevitable.
[1]: http://online.wsj.com/news/articles/SB1000142412788732342060...
That's a nice theory, but the impression I get from the (extra biased against last mile ISPs) DSLReports is that Comcast is continually spreading usage caps and plans to make them nationwide in 5 years, e.g. http://www.dslreports.com/shownews/128987
Lots more with this search: http://www.dslreports.com/nsearch?cat=news&q=comcast%20caps
Lots more with this search: http://www.dslreports.com/nsearch?cat=news&q=comcast%20caps
There is no inherent problem with usage capped plans, as long as the price is fair. If you'd like an uncapped plan, pay more.
The problem is that the last-mile providers try to extract money from netflix for a service that the consumer already paid for (deliver those video bytes)
The problem is that the last-mile providers try to extract money from netflix for a service that the consumer already paid for (deliver those video bytes)
You don't see a problem with e.g. Comcast implicitly charging the consumer extra for using Netflix (by exceeding the cap and paying by the drink afterwords) and not charging anything "extra" for using Comcast's video services?
There is an argument there, in that it costs more real money to deliver video bits the a la carte Netflix way than the cableco broadcast way, but I believe the conflict of interest remains an issue, and the prices I see for exceeding these rather small caps don't strike me as fair.
(Albeit AT&T's, the only choice I have aside from a not so reliable WISP, are particularly ridiculous: 150 GiB/month including I'm not sure what overhead for a continually rising price 2nd from the bottom "up to" 1.5 Mbs down/300+ Kbs up line that currently costs $36/month, each additional 50 GiB costs $10. We'd get a faster line so my father could watch video, at $5/month extra each increment, if the cap wasn't so low and the overages so high.).
There is an argument there, in that it costs more real money to deliver video bits the a la carte Netflix way than the cableco broadcast way, but I believe the conflict of interest remains an issue, and the prices I see for exceeding these rather small caps don't strike me as fair.
(Albeit AT&T's, the only choice I have aside from a not so reliable WISP, are particularly ridiculous: 150 GiB/month including I'm not sure what overhead for a continually rising price 2nd from the bottom "up to" 1.5 Mbs down/300+ Kbs up line that currently costs $36/month, each additional 50 GiB costs $10. We'd get a faster line so my father could watch video, at $5/month extra each increment, if the cap wasn't so low and the overages so high.).
That's not a capped plan. A capped plan charges you more if you exceed a specific bandwidth threshold - no matter what you spend the bits on. As long as Netflix, Comcast PPV, Youtube and my private cat videos get charged the same price - I'm fine with that.
The issue you're pointing out is the lack of net neutrality - but that is orthogonal to the pricing.
The issue you're pointing out is the lack of net neutrality - but that is orthogonal to the pricing.
When it's cheaper for them to host the caches a service offers than increasing their peering bandwidth. That's a fairly simple exercise for controlling.
When the space and electricity for the hosting is cheaper than the transit costs they would otherwise incur.
It's not that they're providing free hosting. Netflix is providing hardware that ISPs can install in their own network locations (which they typically have plenty of). In exchange, their customers get better quality and ISPs don't spend their uplinks on Netflix content. It's pretty win-win for everyone.
Your question is backwards. ISPs can choose this option to save themselves money.
I suspect that the real reason that ISPs like Verizon are resisting, is that with open connect in place there would be nowhere left to hide and blame the internal ISP bandwidth problems.
I suspect that the real reason that ISPs like Verizon are resisting, is that with open connect in place there would be nowhere left to hide and blame the internal ISP bandwidth problems.
It's not hosting. These devices are not available to the open internet, only customers of the ISP who's network they are on.
And I'd say the threshold is the point at which you are no longer to deliver the content by other means.
And I'd say the threshold is the point at which you are no longer to deliver the content by other means.
First, you'd have to buy and build a server for each ISP containing that content and support it when things go wrong. Next, it would have to be cheaper for the ISP to host your server on their network than for them to upgrade their connections to the backbone to handle traffic to your site.
If those two conditions are true, I am sure ISPs will accept your offer.
If those two conditions are true, I am sure ISPs will accept your offer.
Possibly because your homepage does not drive a third of Internet traffic, both from within the ISPs network and from without.
The Internet is a collection of public and private networks. When it costs more to upgrade the linkages between your network and everyone else's than to host a server inside it, that is when the ISP should pay to support a local resource.
You're comparing apples and oranges, really. If your website ever reaches the point where traffic starts to saturate an ISP's connections, I'm sure they would be willing to discuss a local CDN resource with you, too. Whether they offer to pay for that or charge you for it depends largely upon how stupid and/or greedy their management is.
The Internet is a collection of public and private networks. When it costs more to upgrade the linkages between your network and everyone else's than to host a server inside it, that is when the ISP should pay to support a local resource.
You're comparing apples and oranges, really. If your website ever reaches the point where traffic starts to saturate an ISP's connections, I'm sure they would be willing to discuss a local CDN resource with you, too. Whether they offer to pay for that or charge you for it depends largely upon how stupid and/or greedy their management is.
As I understand it, this is similar to one of the ways Akamai builds out it's network:
http://www.akamai.com/html/partners/network_partner.html
http://www.akamai.com/html/partners/network_partner.html
From your phrasing you work at Netflix right? Is it possible to get a full list of your Open Connect partners in a marketplace? In the competitive UK marketplace this could pretty easily be the difference in a purchasing decision for me.
Do you guys use IPv6 at all with your server/client connections?
www.netflix.com/openconnect
They also offer free hardware that caches and serves up the most popular content automagically.
[)amien
[)amien
I don't know how Netflix figures out that it's Verizon that's slow, but lets say it's with some software that anyone could use. Wouldn't it benefit the entire Internet to open source that software so that anyone could use it? Or, barring that, provide an API that anyone can query.
This way, any company can use the software and report the same thing to their customers. Lower the barrier to entry for Amazon/Google/Hulu/your startup to join the fight and inform the people.
This way, any company can use the software and report the same thing to their customers. Lower the barrier to entry for Amazon/Google/Hulu/your startup to join the fight and inform the people.
mtr
probably over time, comparing transit during and not during peak viewing times for a given isp
probably over time, comparing transit during and not during peak viewing times for a given isp
Maybe I'm nitpicking, but in their bridge analogy, wouldn't the drivers be more analogous to the users than to Netflix? If anything, Netflix would be the city full of employment opportunities that drivers are commuting to and from across the bridge.
So a more fitting comparison would be 'like blaming the city at the other end of the bridge for traffic jams..'
Either way, brilliant response, and hopefully Verizon gets the hint.
Either way, brilliant response, and hopefully Verizon gets the hint.
I thought it was a reference to the Gov. Chris Christie bridge scandal, alluding to Verizon's corrupt nature.
Most definitely. But Verizon hasn't quite gone so far as blaming it's own customers for using Netflix, just blaming Netflix for attracting so much traffic.
Maybe you're just trying to read too much into the analogy. I think Netflix is basically saying that the infrastructure is there (the bridge), but despite the fact that usage has increased (rush hour), Verizon is artificially capping speed (keeping lanes closed) to its users (the cars) by demanding that Netflix pay extra for Verizon to open more lanes on the bridge.
Yes, I certainly read too much into it. Just pointing out a flaw in the analogy. Verizon demanding that Netflix pay extra for users to access its content would be more like the Christie demanding the city of New York to pay for more lanes to be opened on the bridge. While simultaneously charging the drivers (the users) for access to it.
Analogies are always flawed. Must we really continue to dive deep into every one and point out all the flaws, or can we start to just read them at the face value they were written at and move on?
The distinction is that in your proposed scenario the third parties are the ones deciding whether or not to travel across the bridge, so the only thing the cities can do is try to anticipate bridge usage and make sure they have enough lanes.
With the Verizon/Netflix relationship, Verizon upsells higher-speed plans by advertising better speeds for online services like Netflix. They already sell high-speed connections to their customers with advertised speeds far in excess of what you'll actually get when using a service like Netflix. So it is a bit intrinsically dishonest for them to try and pin blame for service congestion on Netflix when (generally speaking) other ISPs manage to deliver much higher connection speeds, closer to what customers are actually paying for.
So a better bridge analogue in this scenario would be if people paid up front for access to the bridge. Some people might pay for the bridge and use it rarely, other people might pay for the bridge and use it all the time. Anyone who pays for bridge access and uses it every day can rightly expect that they should be able to get across the bridge, and the bridge operators can try to cut costs by estimating actual bridge usage and investing a little bit less in lanes & other services. If they mess up their estimates, the people who pay to use full time will be delayed and get frustrated.
With the Verizon/Netflix relationship, Verizon upsells higher-speed plans by advertising better speeds for online services like Netflix. They already sell high-speed connections to their customers with advertised speeds far in excess of what you'll actually get when using a service like Netflix. So it is a bit intrinsically dishonest for them to try and pin blame for service congestion on Netflix when (generally speaking) other ISPs manage to deliver much higher connection speeds, closer to what customers are actually paying for.
So a better bridge analogue in this scenario would be if people paid up front for access to the bridge. Some people might pay for the bridge and use it rarely, other people might pay for the bridge and use it all the time. Anyone who pays for bridge access and uses it every day can rightly expect that they should be able to get across the bridge, and the bridge operators can try to cut costs by estimating actual bridge usage and investing a little bit less in lanes & other services. If they mess up their estimates, the people who pay to use full time will be delayed and get frustrated.
They might as well have addressed that letter to hacker news with a cc to verizon. Well executed PR move.
What I find most interesting about these topics is how many people come out to contribute to them. Nothing gets people riled up like vaguely threatening the free flow of entertainment.
I wish I know how to direct this energy into something a little less disappointing.
I wish I know how to direct this energy into something a little less disappointing.
Netflix should just stop serving to Verizon and Comcast. the outlash would force the FCC and companies to recognize.
That would be a very bad idea for Netflix. They would be breaching contract on thousands and thousands of customers and open them self to probably a very large class action lawsuit. Although I could be wrong IANAL.
While I agree this is a bad idea---the whole incident shows they have a direct link to the hearts and minds of their subscribers using these ISPs and are not afraid of using it (the biggest development in the net neutrality fight since AT&T's Ed Whitacre opened his foolish mouth and started it 8 years ago)----they could punt subscribers on these ISPs by simply not renewing subscriptions as they finish.
I'm sure they're not representing they'll be providing their service forever, and e.g. issues with content providers means they can't even promise they'll be supplying access to any particular stuff beyond what they outright buy or produce themselves.
I'm sure they're not representing they'll be providing their service forever, and e.g. issues with content providers means they can't even promise they'll be supplying access to any particular stuff beyond what they outright buy or produce themselves.
Maybe it will, or maybe it won't. And if it will, it may come after Netflix' US business is irrevocably damaged. It's not a sane choice for Netflix to make.
doesnt this reek of hoax to anyone? Not just the 'response' letter, but the initial twitter 'screen shot' as well. Why in the world would an executive scan a letter like that opening them up to litigation? even if they were 'right', they would probably get fired for it.
I don't think they scanned it. The scribd account, at least, shows other similar documents uploaded by the same user, none of which were original up to now. I've no reason to doubt its authenticity. I bet it just spread outside the company by people who are interested in net neutrality.
Is this just a USA problem?
Why for example has SKY or TalkTalk in the UK not tried this? What is stopping them?
Why for example has SKY or TalkTalk in the UK not tried this? What is stopping them?
Sky and TalkTalk both rather like having customers, and would be unhappy if any significant number of customers perceived Sky or Talktalk's service as being poor, and chose to move to a different ISP because of it.
UK ISPs do/have used streaming video and buffering (or the absence of it) as a battleground of quality - you see companies making claims of fastest or "most reliable" broadband, and it gets them customers as a result.
Mucking up people's ability to access Netflix would bring no benefit - people would just choose a different ISP.
At least in the UK, where people have that choice. This situation may not apply in other countries.
UK ISPs do/have used streaming video and buffering (or the absence of it) as a battleground of quality - you see companies making claims of fastest or "most reliable" broadband, and it gets them customers as a result.
Mucking up people's ability to access Netflix would bring no benefit - people would just choose a different ISP.
At least in the UK, where people have that choice. This situation may not apply in other countries.
Google needs to bring its ISP services to all US markets faster. Kill these shitty/pos ISPs. I'm really hoping Google is willing to throw enough capital at this problem and change some of the laws to open up the market to all new comers.
I'm terrified of Google (or any single player) owning the entire stack from infrastructure to devices to services. Here's to hoping that Google Fiber (and municipal fiber) spur competition, rather than a new monopoly.
I'm all for what Netflix is doing, but this read like a cheesy publicity stunt. The whole thing seems like it was written to be read by the public, not Verizon's lawyers. I mean, it contained a simile/metaphor.
Ideally everything legal entities write, that can be written for the public, should be written for the public. And when I say "for the public" I mean in terms the public can understand. So yes, including simile/metaphor.
Since the public isn't really up to speed on the net neutrality issue, I can't think of a better way to educate them.
This war is going to be fought on multiple fronts, and public relations is a very large one.
This war is going to be fought on multiple fronts, and public relations is a very large one.
It's becoming a habit that every week either telco responds to Netflix, or Netflix responds to telco.
I'm slightly sick of these silly public letters. Why the hell either Netflix or said telcos feel like they need to bother the public with their internal problems, instead of solving them privately like adults?
This is not about net neutrality. It's about who foots the bill for upgrading the network to handle the capacity that Netflix users need.
Meter the damn traffic and let Netflix (and other heavy) users pay the telco and be done with it. God damn.
I'm slightly sick of these silly public letters. Why the hell either Netflix or said telcos feel like they need to bother the public with their internal problems, instead of solving them privately like adults?
This is not about net neutrality. It's about who foots the bill for upgrading the network to handle the capacity that Netflix users need.
Meter the damn traffic and let Netflix (and other heavy) users pay the telco and be done with it. God damn.
Is it only me, who thinks, this letter might be FAKE. Though it's professionally written but it doesn't look like real one.
For example: three DOTs right after word doorstep. I tend to use multiple dots in my personal emails but even in office, talking to clients, using multiple dots, is not professional.
How we can confirm, this is REAL letter not just nice job done faking it.
The three dots is the equivalent of an ellipsis. There's no reason for that to be unprofessional.
http://en.wikipedia.org/wiki/Ellipsis
Now, random capitalization...
Now, random capitalization...
The lack of any response from Netflix disclaiming the letter as a fake seems to speak volumes. Also http://grammar.ccc.commnet.edu/grammar/marks/ellipsis.htm
I'm not sure why you are getting downvoted, but I thought the same thing. Ellipses aside, it does not read like a letter written by an attorney.
oh grammer nazis... :). I thought, it's only reddit.
Point is, it doesn't look "legal" enough language. Again, I just have doubt on it's legality.