(Native German speaker here), it's a very rare use of the word "free" and usually only used in fixed terms like "Freibier", it wouldn't even work for other drinks, e.g., you can't say "Freisaft" or "freier Saft" for free juice, it has to be kostenlos there.
The other Wiktionary example of "freie Krankenversorgung" sounds wrong to be, but it seems to be used rarely in some more formal or legal contexts, no one would say it like this in a casual conversation. Google results also show a 4x difference between frei and kostenlos here in favor of kostenlos. But both are low since "Krankenversorgung" is already a very unusual word. I suspect many of those uses might be bad translations from English.
I'm surprised that the recent advances in applying typography to engineering problems [1, 2] are not published at SIGBOVIK but are apparently going to a more serious journal.
We wrote some experimental network drivers in high-level languages a few years ago and Java performed better than expected. Yeah, that's a very special and somewhat odd use case, but it was fun :)
We've got some graphs [0] comparing throughput and long-tail latency for the various Java GCs available in OpenJDK 12. Shenandoah's worst case pause time was ~45µs which is the same as disabling the GC (Epsilon GC) which is pretty impressive. Overall performance did suffer under Shenandoah a little bit back then, though. However, I've heard that this improved recently.
If you want empiricism try academic papers instead of mailing lists ;) Some numbers: most critical bugs in the Linux kernel are due to memory safety: 40 out of 65 bugs allowing for code execution found in Linux in 2017 could have been prevented by using a memory-safe language [0]. The paper is about Go, but the same logic applies for Rust. Fun fact: 39 out of these 40 bugs were in drivers [1], so I think starting with drivers in Rust is a good idea.
The real root cause for all that flow director mess and core balancing is that there's a huge disconnect between how the hardware works and what the socket API offers by default.
The scaling model of the hardware is rather simple: hash over packet headers and assign a queue based on this. And each queue should be pinned to a core by pinning the interrupts, so you got easy flow-level scaling. That's called RSS. It's simple and effective.
What it means is: the hardware decides which core handles which flow. I wonder why the article doesn't mention RSS at all?
Now the socket API works in a different way: your application decides which core handles which socket and hence which flow. So you get cache misses if you don't tak into account how the hardware is hashing your flows. That's bad. So you can do some work-arounds by using flow director to explicitly redirect flows to cores that handle things but that's just not really an elegant solution (and the flow director lookup tables are small-ish).
I didn't follow kernel development regarding this recently, but there should be some APIs to get a mapping from a connection tuple to the core it gets hashed to on RX (hash function should be standardized to Toeplitz IIRC, the exact details on which fields and how they are put into the function are somewhat hardware- and driver-specific but usually configurable). So you'd need to take this information into account when scheduling your connections to cores. If you do that you don't get any cache misses and don't need to rely on the limited capabilities of explicit per-flow steering.
Note that this problem will mostly go away once TAPS finally replaces BSD sockets :)
Lesson 1: Never ever reboot multiple Ceph nodes without checking if Ceph is happy between reboots. This failure happened early during boot and this could have been handled with no downtime if they checked the rebooted nodes before rebooting the next one.
Lesson 2: Avoid using RAID controllers except for the most simple "pass through" mode.
ceph-volume finally fully removed dependency on file systems. Yeah the LVM-mess is sometimes annoying and early version of ceph-volume had many problems, but nowadays I wouldn't want ceph-disk back.
The NIC can hash on IPs and UDP ports just fine, nothing to worry about here.
Sure, there's the rare case of a connection moving to a different L3/4 flow identifier causing a cache miss, but that's not something that happens often enough to be a relevant bottleneck.
The hardware selection and test setup is questionable, yeah. But downclocking the CPU is pretty standard for many benchmarks like that.
Would have been better to evaluate the effects of CPU frequency changes separately. But there's nothing wrong with downclocking in general if you want to evaluate effects of CPU bottlenecks when you don't have high-speed links available.
Isn't a large part of the performance gain from TSO offloading on transmit due to avoiding working with way too many skb's in the kernel?
Shouldn't GSO for UDP with proper NIC support give you most of the performance gain?
I'm not sure about the implementation details of GSO and TSO in Linux. But I know a thing or two about the hardware: the Intel Niantic/82599 NICs support UDP transmit segmentation offloading, I've checked the datasheet real quick and it says "Note that current UDP segmentation offload is not supported by any standard OS."
Not sure if this is just an outdated comment in a 10 year old datasheet or if there's some performance to be gained by using this UDP segmentation offload for quick.
Edit: Should have clicked on the link in the article [1] before posting here, UDP segementation offloading is coming to lots of NIC drivers in 5.5.
Always interesting how the hardware often supports way more features than the drivers expose and you suddenly get new hardware features on a 10+ year old NIC model :)
(Similar story with IPsec offloading on the Niantic/82599)
> But enabling tcp timestamps in general case brings little benefit and wastes 12 bytes of each packet for basically no gain.
I disagree; TCP timestamps are awesome. Linux enables these by defaults.
Quick search gives me some measurements from 2012 [1] that indicate that TCP timestamps are enabled on 83% of the top 100k web hosts.
You can afford to waste 12 bytes; the bottleneck isn't these 12 bytes but how well you get congestion control to work. And congestion control relies on getting an accurate estimate of the round-trip time
Edit: Also, just because 83% of web hosts having it enabled does not imply that it is a good idea to do so in general. They could just all be running the linux defaults and these could be just wrong
There's only one level of TLBs in the IOMMU. And that's 64 entries.
Yeah, I think the dTLB is only 64 entries on Intel CPUs as well, but there's a second larger layer behind that, and an even larger third layer. IIRC it's a total of 4096 entries on recent Intel CPUs.
I think ~100k to 200k TSO "packets" per second should be doable with the IOMMU. But I guess it depends where the data is coming from. Could be one of the odd cases where copying data is faster than doing zero-copy, e.g., just copy everything into the same small set of small-ish buffers to keep the number of pages that need to be present in the IOMMU small?
A problem on the hardware side is that Intel's IOMMU TLB is tiny (64 entries), so using huge pages for all DMA-accessible memory is absolutely required to get a good performance out of it.
Not a game developer, but I used to write UI addons for World of Warcraft. WoW allows you to customize your UI heavily with these Lua plugins ("addons") and Lua is garbage collected. It's a reasonable incremental GC so it shouldn't be too bad in theory.
But in practice it can be horrible. You end up writing all kinds of weird code just to avoid allocations in certain situations. And yeah, GC pauses due to having lots of addons is definitely very noticable for players.
Also leads to fun witch hunts on addons using "too much" memory, people consider a few megabytes a lot because they confuse high memory usage with high allocation rate... Our stuff started out as "lightweight" but it grew over the years. We are probably at over 5 MB of memory with Deadly Boss Mods (many WoW players can attest that it's certainly not considered lightweight nowadays, but I did try to keep it low back then).
But I think we still do a reasonable job at avoiding allocations and recycling objects...
The point is: I had to spent a lot of time thinking about memory allocations and avoiding them in a system that promised me to handle all that stuff for me. Frame drops are very noticable.
But there are languages with better GCs than Lua out there...
The other Wiktionary example of "freie Krankenversorgung" sounds wrong to be, but it seems to be used rarely in some more formal or legal contexts, no one would say it like this in a casual conversation. Google results also show a 4x difference between frei and kostenlos here in favor of kostenlos. But both are low since "Krankenversorgung" is already a very unusual word. I suspect many of those uses might be bad translations from English.