The linked article says they decided to do CD in 2016 fwiw so that's not inconsistent with what I said.
You reduced the number of patches a lot and also pushed very hard to get us to 3.0 after we sat on 2.6.38 ~forever. Which was very appreciated, btw. We built the whole plan going forward based on this work.
I'm not arguing that anyone should be nice to anyone or not (it's a waste of breath when it comes to Linux). I'm just saying that the benchmarking was thorough and that contemporary 2014 hardware could zero pages fast.
For the peanut gallery more: I worked with both of these guys at Meta on this.
The "servers are only on for a few hours" thing was like never true so I have no idea where that claim is coming from. The web performance test took more than a few hours to run alone and we had way more aggressive soaks for other workloads.
My recollection was that "write zeroes" just became a cheaper operation between '12 and '14.
A fun fact to distract from the awkwardness: a lot of the kernel work done in the early days was exceedingly scrappy. The port mapping stuff for memcached UDP before SO_REUSEPORT for example. FB binaries couldn't even run on vanilla linux a lot of the time. Over the next several years we put a TON of effort in getting as close to mainline as possible and now Meta is one of the biggest drivers of Linux development.
It was definitely a toy, I transliterated from python bytecode (a stack based vm) into bpf. I also wrote the full code gen stack myself (bpf was simpler back then)
But using llvm and not marrying things to cpython implementation makes this approach way better
The cloud business model is to use scale and customer ownership to crush hardware margins to dust. They’re also building their own accelerators to try to cut Nvidia out altogether.
The thing that we need in order for your dream to become a reality is excellent user space frameworks, so I encourage you (and anyone else) to go build one or (better) find one you like and contribute.
> File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads.
AIO has existed for a long time. A lot longer than io_uring.
I think the thing that the author misses here is that the majority of IO that happens is actually interrupt driven in the first place, so async io is always going to be the more efficient approach.
The author also misses that scheduling threads efficiently from a kernel context is really hard. Async io also confers a benefit in terms of “data scheduling.” This is more relevant for workloads like memcached.
Age is great. I used the rust crate to write an ftp server that encrypts the files before they hit disk (specific use case is having a drop box for my network scanner) and I love the simplicity and composability it provides.
One feature request: it would be awesome to have paraphrase encryption for age private keys.
The core framework (sched_ext) was written for general workloads and can quite beneficial. It lowers the costs of creating and iterating on schedulers quite a bit.
To be honest, when they started working on it I don’t think any of us expected for it to be a source of collaboration with gaming companies :)
Latency and cache coherency are the other things that make this hard. Cache coherency can theoretically be resolved by CXL, so maybe we’ll get there that way.
A professor actually told us that freeing prior to exit was harmful, because you may spend all of that time resurrecting swapped pages for no real benefit.
Counterpoint is that debugging leaks is ~hopeless unless you have the ability to prune “intentional leaks” at exit
OpenAI's recruiting pitch was 5-10+ million/year in the form of equity. The structure of the grants is super weird by traditional big-company standards, but it was plausible enough that you could squint and call it the same. I'd posit that many of the people jumping to OpenAI are doing it for the cash and not the mission.
You reduced the number of patches a lot and also pushed very hard to get us to 3.0 after we sat on 2.6.38 ~forever. Which was very appreciated, btw. We built the whole plan going forward based on this work.
I'm not arguing that anyone should be nice to anyone or not (it's a waste of breath when it comes to Linux). I'm just saying that the benchmarking was thorough and that contemporary 2014 hardware could zero pages fast.