The name "bufferbloat" is falling out of favor. First off, it has a horrible sound... And it doesn't really give an intuitive sense of what's wrong.
Referring to "latency" or (my favorite) "responsiveness" is better.
And I was encouraged to see this recent ZDNet article that mentioned the "ping rates" of 600-1000 msec, and notes that these would cause videoconferencing or gaming to be unusable.
> I just stopped updating the controller software (none of their gear is external-facing, and IIRC it's only needed for configuration/management) because cloud login is an absolute dealbreaker for me.
Yeah. Updates used to be a nightmare. I had to worry about Windows updates, Java updates, and of course Unifi updates.
I have 21 APs all controlled by the container on a Raspberry Pi 4. It's not even breaking a sweat. When I want to upgrade the Unifi application, I stop the container, and re-run the command to use the newer Unifi version. Three minutes later, it's back on the air.
As another poster noted, the Feds funded a lot of the cost of electrification. There is starting to be some action on the internet front. But...
Here are the economics for a fiber optics deployment in 2021 for a rural town of ~800 premises, where about 60% got service. They charge customers about $100/month for symmetric 75/75mbps internet plus phone.
It costs about $40K per mile to run the fiber down the road on existing utility poles. It costs between $2K and $4K to run the drop from the utility pole to the home.
Those two numbers (miles of road, number of premises) get you in the ballpark of the cost of deployment in a town/county. More premises per mile obviously decreases the cost per premise - that's why utilities prefer dense areas (cities).
@LinuxBender is on the right track. Bufferbloat can cause this kind of lag. (There may be other causes, but first thing to do is check for bufferbloat.) To do this use:
This measures the latency/lag while data is being transferred. When it's complete, click "Results + Share" and you'll see a report. Send us the link of that page so we can see your results.
Your router is holding ("buffering") packets in the hopes that they can be sent soon. Your measurements indicate that the router is "bloated", holding about five seconds (5,000 ms) worth of data.
This gives the sending TCP algorithm the wrong impression. It's waiting to hear about a dropped packet to indicate that there's congestion. When your router holds on to those packets (instead of dropping them), the TCP algorithm doesn't get any feedback, so it keeps shoveling data into the connection.
This leads to the bad state you're seeing. And that's where the advice on "What can I do about Bufferbloat?" comes in.
There's no benefit to having more than one packet buffered by the router. (Hanging on to more than one packet per connection only causes the latency/lag you're seeing.)
There are routers that actually check the time the router has held packets. If packets have been queued for "too long", the router discards them immediately, giving the vital feedback to the sending TCP. Those routers use the technique known as SQM (Smart Queue Management) and the fq_codel, cake, PIE algorithms to keep the queues within the router short - typically less than 5 msec.
To solve your problem, investigate getting a router that implements one of those SQM algorithms. They're listed on the "What can I do..." page. I am a fan of OpenWrt (use it at home), but have installed a bunch of IQrouters and Ubuquiti devices for friends.
Please report back to the list so we can refine our knowledge. (It's like Covid-19 testing. We cannot know the incidence of infection if we don't test lots and lots of people to find a true rate...) Thanks.
> Why should the video bitrate be dropped because some second rate ISPs are oversubscribing super badly? Isn't all of this totally against the principles of net neutrality?
No. This is entirely separate from net neutrality. That principle says, "No ISP should discriminate on the basis of content."
In the case of an oversubscribed ISP, the "magic of the markets" that so many lobbyists talk about is that you can simply switch to a new provider. Oh, you only have one provider? Tough cookies.
Netflix (and Youtube and Hulu and all the other big content providers) pay big bucks (millions per month) to get the content from their servers to the internet backbone. Their contracts with their ISPs don't specify where the traffic goes. They just buy lots of 10 gpbs links, and send the data off. They're paying their share.
Your contract with your ISP doesn't distinguish between getting traffic from a mom&pop website or Netflix. It just said they would deliver the rated speed (7 mpbs or 10mbps, or 100mbs) of data to you for your fixed monthly fee.
When the ISPs found out that people were actually using that much data (that YOU PAID FOR), they found they had underprovisioned their network, and couldn't support the load. ISPs started saying, "Netflix is using too much", because, really, it's rude to blame your customer for using the service you provide as contracted.
This is called double-dipping.
So - Netflix is paying their fair share. You are paying your fair share. If your ISP can't handle the traffic demands of their customers, they need to suck it up and provide the capacity that you're paying for.
Latency is a measure of how long it takes a packet to go from your computer to some other computer and back. People usually measure this by a ping test
Unloaded latency (when there isn't much traffic on your connection through your link to the ISP) generally ranges from 5 msec to 50 msec, depending on what kind of connection you have (fiber, cable, dsl, cellular).
Loaded latency is measured when there is traffic on the link. A good router will minimize the additional latency, but many routers will add a lot - between two and five seconds (2000 to 5000 msec). That's why you lag out in games.
> In either case, neither queue builds up in the home router -- that is exactly the problem, since we can't control the queue size in the modem or ISP! But by traffic shaping to 3.25/0.85 Mbps in the home router (either manually or automatically with fq-codel), we force the queues to build up there, thus giving us control.
YES! I understand what you're saying. (I had been treating the cable/DSL links and their terminating equipment as a black box; your description was being far more specific. Thanks for sticking with me to make yourself clear.)
And as you point out, the fix in all cases (at least, until our ISPs get serious about solving bufferbloat) is to force the home router to take control of the buffering by shaping the traffic a bit below the actual link speed. Thanks.
Yes, but... A "bottleneck" occurs wherever there is a high speed link going to a lower speed link. In your case, the 100 Mbps Ethernets in your home feed through the router to the 1 Mbps upstream your ISP provides. That's another bottleneck.
At this point in a conversation, I always recommend people measure their actual network, to see if they're happy with the situation. If it's good, then everyone's happy.
What results to you get from the DSLReports Speed Test (www.dslreports.com/speedtest)? It measures latency (lag) during the download and upload parts of the test, and will show if your router (or your ISP's router) is buffering too much data, and giving you undesired latency. Best regards.
a) I reviewed your eur-lex.europa.eu link, and didn't find anything that prohibits installing other firmware. Please cite the chapter/article that's relevant.
b) re: TP-Link firmware. It is true that TP-Link has added a technical means that makes it impossible to install certain other TP-Link firmware (the "downgrade" they mention). However, there are other firmware distributions (LEDE/OpenWrt/DD-WRT/others) that have the capability to install over the TP-Link factory firmware.
I'm sorry, but I have to disagree. The laws of physics (of the internet) dictate that the piece of equipment at a bottleneck is responsible for handling bloat/congestion on its own end. Your ISP's head end/DSLAM/etc. has an opportunity to prevent download bloat (with data being sent to the home), but it has no way to know what's happening on the consumer end of the link.
Your router at home needs to have the same logic: it needs to control the buffering of data being sent toward the ISP. fq_codel/cake actually measures the time packets dwell in the queues (their "sojourn time") and drops/marks with ECN some of the packets of flows that build up a queue.
A cool feature of fq_codel/cake qdisc in LEDE is that it can control bloat in both directions. It imposes a bottleneck that's slightly (a few percent) below the actual ISP download link speed so that traffic queues up within your home router (instead of at the DSLAM/head end). Consequently, it can do the fq_codel/cake algorithm on the download (as well as the upload) direction, keeping your link unbloated at a very small loss of link speed.
And there's good news on the Wi-Fi front as well. Many of the same people who brought us fq_codel/cake have new algorithms for "Making Wi-Fi Fast".
Their paper tells how they've reduced bufferbloat as well as increasing "airtime fairness" in the Wi-Fi stack. The paper, "Ending the Anomaly: Achieving Low Latency and Airtime Fairness in WiFi", is now available as a preprint at: https://arxiv.org/abs/1703.00064 The conclusion of the paper states:
"We have developed a novel two-part solution to two large performance problems affecting WiFi: Bufferbloat and the 802.11 performance anomaly.
"The solution consists of a new integrated queueing scheme tailored specifically to eliminating bufferbloat in WiFi, which reduces latency under load by an order of magnitude. Leveraging the queueing structure, we have developed a deficit based airtime fairness scheduler that works at the access point with no client modifications, and achieves close to perfect fairness in all the evaluated scenarios, increasing total throughput by up to a factor of 5.
"Our solution reduces implementation complexity and increases accuracy compared to previous work, and has been accepted into the mainline Linux kernel, making it deployable on billions of Linux-based devices."
Fun reading! Oh, and it's available in LEDE today.
Yes, but it can occur on both ends of the bottleneck. Your ISP's equipment has a bottleneck link toward your home, and can have bloated buffers on that side. But if your router doesn't handle the uplink well, it'll be bloated as well.
Fortunately, SQM is available (from LEDE) for many MikroTik routers, so there may be a way out for the GP.
Sure, you could make yourself crazy trying to prioritize all the traffic with varying kinds of flags.
Or you could just let an algorithm (SQM - fq_codel or cake) automatically determine which flows are sending more than their fair share of traffic into the bottleneck link, and offer backpressure to slow those applications down, so other applications work well.
re: net neutrality. That applies to whether my ISP (or any provider upstream in the Internet) should prioritize packets/data from various sources toward (or from) me. The answer should be, No. My contract with my ISP is that I get X mbps of data. The contract doesn't say the ISP will provide higher priority or data rate for service Y over service Z...
SQM has no effect on net neutrality: I'm simply configuring my own router to prioritize the data that I'm requesting (or sending). My ISP should handle them in a totally neutral way...
I already do this. But I cheat - I use a good router (OpenWrt One) that has built-in controls for Bufferbloat. See [How OpenWrt Vanquishes Bufferbloat](https://forum.openwrt.org/t/how-openwrt-vanquishes-bufferblo...)