> though going the in-house approach would warrant keeping an eye on the relevant kernel commits like a hawk to avoid missing bug fixes like these. These in-house implementations tend to have less eyeballs than the kernel.
This is somewhat funny to read because this specific issue in CUBIC (sudden CWND jump upon existing quiescence) was originally discovered in Google's QUIC library and then later reported to the team working on the TCP stack. I know this because I was the one who found that bug back in 2015.
That said, congestion control algorithms are really prone to logic bugs, and very subtle changes in the algorithm can often lead to dramatically different outcomes. Because of that, there's a lot of value in running congestion control code that has been tested on a wide variety of real Internet traffic.
The story here is a bit complicated. WebTransport is, in some sense, an evolution of RTCQuicTransport API, which was originally meant to solve the issues people had with SCTP/DTLS stack used by RTCDataChannel. At some point, the focus switched to client-server use cases, with an agreement that we can come back to the P2P scenario after we solve the client-server one.
This sounds very similar to how base::WeakPtr works in Chromium [0]. It's a reasonable design, but it only works as long as the pointer is only accessed from the same thread it is created.
The article seems to make an assumption that the application backend is in the same datacenter as the load balancer, which is not necessarily true: people often put their load balancers at the network edge (which helps reduce latency when the response is cached), or just outsource those to a CDN vendor.
> In addition to the low roundtrip time, the connections between your load balancer and application server likely have a very long lifetime, hence don’t suffer from TCP slow start as much, and that’s assuming your operating system hasn’t been tuned to disable slow start entirely, which is very common on servers.
A single HTTP/1.1 connection can only process one request at a time (unless you attempt HTTP pipelining), so if you have N persistent TCP connections to the backend, you can only handle N concurrent requests. Since all of those connections are long-lived and are sending at the same time, if you make N very large, you will eventually run into TCP congestion control convergence issues.
Also, I don't understand why the author believes HTTP/2 is less debuggable than HTTP/1; curl and Wireshark work equally well with both.
Isn't this the problem that JSON5 (and probably other similar projects) is supposed to solve?
Both JSON (as defined in the RFC) and JSON5 have a nice property of being well-defined, meaning that you can use different libraries in different languages on different platforms to parse them, and expect the same result. "JSON but parser behaves reasonably (as defined by the speaker)" does not have this property.
> HTTP → RFC-2616 says in section 19.3 says "we recommend that applications ... recognize a single LF as a line terminator...." In other words it is perfectly OK for an HTTP client or server to accept CR-less HTTP requests or replies. It is not a violation of the HTTP standard to do so. Therefore they should.
The most up-to-date version of HTTP/1.1 spec is RFC 9112, which says:
> Although the line terminator for the start-line and fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR.
"MAY", of course, is different from "MUST" or "SHOULD", so I feel like the author's claim that implementations rejecting bare NLs are broken is at odds with the specification.
W3C generally requires Working Group participants to provide IPR licensing commitments for the spec in question [0]. As far as I understand, higher level of specification maturity implies stronger level of obligations, though the specifics of what specifically changes when were never clear to me.
Generally, L2 networks are engineered with the assumption that they will carry TCP, and TCP performs really poorly with high loss rates (depends on the specific congestion control used, but the boundary can be anywhere between 1% and 25%), so they try to make sure on L2 level that losses are minimal. There are some scenarios in which a network can be engineered around high loss rates (e.g. some data center networks), but those don't use TCP, at least with traditional loss recovery.
Error correction codes on L4 level are generally only useful for very low latency situations, since if you can wait for one RTT, you can just have the original sender retransmit the exact packets that got lost, which is inherently more efficient than any ECC.
It's possible to build something similar on top of TCP, see Minion [0] for an example. There are multiple reasons why this is less practical than building on top of UDP, the main two being, from my perspective: (1) this requires cooperation from the OS (either in form of giving you advanced API or having high enough privilege level to write TCP manually), and (2) this falls apart in presence of TCP middleboxes.
QUIC requires the initial handshake packets to be at least 1200 bytes, and sets the anti-amplification limit of 3x [0]. This means that the server can typically send up to 3600 bytes in response (unless the client's handshake message exceeds one packet, which usually only happens if there is a post-quantum key share in it). 3600 bytes is usually enough, unless your certificate chain is too large, in which case you'd need to compress it. [1] is a nice overview of the problem.
> There seems to be not too much interest in or hype about WebTransport.
I feel like this is partially because the main value of the API comes from working better on lower-quality networks, rather than providing the ability to do something completely brand new.
> FF and Safari don't support it yet.
For what it's worth, Mozilla are positive about it [0] and are actively involved in the standards process. I don't believe the WebKit team has stated their position, though there are Apple people who are involved in the standards process too.
> I think it's missing a killer app. But I am not sure what WebTransport would enable that isn't already possible with current APIs.
When we started out working on WebTransport, there were two major use cases we had in mind: live media and web games. IETF MoQ working group [1] will hopefully address the first one. I don't know whether anyone is using it for building games, but I don't actually have that much visibility into what's going on in that space.
The lack of good examples mostly stems from the fact that there isn't really that many publicly available server libraries yet. The two I can name off the top of my head are aioquic (Python) and the Google QUIC implementation used in Chromium and ENvoy -- the latter is sadly not trivially embeddable by third-party code (I work on it, and I've been trying to make it easier to use, but given that it's a large C++ codebase with a lot of dependencies, this has been taking a while).
For YouTube, the CPU cost of QUIC is comparable to TCP, though we did spent years optimizing it. [0] has a nice deep dive.
Other CDN vendors like Fastly seem to have the similar experience [1].
I believe sendmmsg combined with UDP GSO (as discussed, for instance, in [2]) has solved most of the problems that were caused by the absence of features like TSO. From what I understand, most of the benefit of TSO comes not from the hardware acceleration, but rather from the fact that it processes multiple packet as one for most of the transmission code path, meaning that all per-packet operations are only invoked once per chunk (as opposed to once per individual IP packet sent).
I am not sure I follow this. WebTransport defines a way to transfer arbitrary data over HTTP/3, it's not a media protocol. I expect that a well-defined media transfer protocol will eventually emerge for media over WebTransport, and it will make its way to IETF.
(FWIW, you can send RTP over WebTransport datagrams, instead of using SRTP and ICE-lite)
If anyone is wondering if this is already implemented anywhere, we're currently experimenting with it in Chrome: https://web.dev/quictransport/ -- I'd be curious to hear what people think about it.
It will never get out-of-sync because TCP guarantees that the bytes will be delivered in the same order they've arrived.
The best approach is typically put a length in front of every message. The good things about that approach are:
1. The receiver can allocate buffer that is exactly the size it needs to fit the message.
2. The receiver can check whether the message is too long before seeing the entire message.
The only disadvantage is that you have to know the length of all messages in advance.
Most of the common TCP congestion control algorithms (Reno, Cubic) are loss-based: they try to send more and more data until the link no longer can buffer all of the packets, and drops some of them. Naturally, this approach requires the buffer to fill up, causing the latency to spike.
There are algorithms that try to use increased delay as a signal that the link is full. This approach has multiple problems, one of which is that delay can be really noisy on wireless networks; another is that if you have a loss-based and a delay-based connection sharing the same link, the delay-based one will get much less than a fair share of its bandwidth. People have been trying to make an algorithm that both coexists with Reno/CUBIC and does not induce bufferbloat for the last 25 years or so, and there's been some progress, but none of it has
reached the point where it could be used as a default congestion control for all operating systems.
The problem of "I have files to transfer in background, but I want my connection to yield to more important traffic" can actually solved using a special congestion control algorithm called LEDBAT [1]; it's used by Apple for things like software updates, and BitTorrent uses it too. Unfortunately, I think only Apple implements it in its TCP stack, so anyone who wants to do that would have to roll their own thing using UDP.
> By the way, does anyone know why QUIC specifically put NewReno as its official congestion control [0]? Different environments benefit from different algorithms so I don't see why.
It's a "reasonable default", in the sense that it normally works quite well, and is easy to implement. The dangerous thing about congestion control is that it's really easy to get it wrong and not notice (as noticing requires performance testing and monitoring).
Google QUIC uses BBR on the server side. I wrote the initial version of said BBR implementation, and it took a lot of effort to get it to a production-ready state, so I would not generally recommend writing BBR from scratch. CUBIC is much simpler than BBR, but is also prone to subtle bugs like [0].
> 2. Not everything that needs encryption needs TLS specifically. TLS is not a one-size-fits-all solution.
I'm actually curious what use cases you have in mind. TLS definitely does not cover 100% of all possible cases, but I've seen a surprising number of cases where people rolled their own thing when using (D)TLS would have just worked.
This is somewhat funny to read because this specific issue in CUBIC (sudden CWND jump upon existing quiescence) was originally discovered in Google's QUIC library and then later reported to the team working on the TCP stack. I know this because I was the one who found that bug back in 2015.
That said, congestion control algorithms are really prone to logic bugs, and very subtle changes in the algorithm can often lead to dramatically different outcomes. Because of that, there's a lot of value in running congestion control code that has been tested on a wide variety of real Internet traffic.