That's what I had in mind! The whole post is a claim that evaluating knowledge work got more expensive because cheaper measures stopped correlating well with quality.
If someone was already evaluating the work output using a metric closer to the underlying quality then it might not have been a big shift for them (other than having much more work to evaluate).
You’re right, there are downsides like turbine you mention! We mitigate it by running a hot backup we can switch to in seconds and a box in which we test restoring backups every 24h, that’s necessary! But it requires 3x the number of big expensive boxes.
I still think it’s the right tradeoff for us, operating a distributed system is also very expensive in terms of dev and ops time, costs are more unpredictable etc.
At $WORK, we write ~100M rows per day and keep years of history, all in a single database. Sure, the box is big, but I have beautiful transactional workloads and no distributed systems to worry about!
Author here. I did not expect to see my post on HN!
It was a rant, I was venting, it’s not supposed to be an objective statement about the state of tech. It’s shouting into the void about the things I find unfair and unbearable, I don’t think it’s a great HN material.
I made up parts of the story because it didn’t happen to me and I didn’t want to share details of somebody else’s situation.
Learning Rust by building a simple database using it.
I’ve done my share of programming languages (PHP, C++, Python, Ruby, Haskell) and for the last 10 years I’ve been working in OCaml (which I love so much) but Rust would be a nice addition IMO.
And I never implemented LSM style database before! So that’s fun.
I think the surprise part in under-appreciated. The quicktest tests I write are frequently of the form “let’s spend 15 minutes and see what happens” kind of tests - and I almost always find bugs that way. The return on investment is bonkers.
I do think that immutable by default OCaml + good PBT tooling there helps a lot.
Even trading companies have a ton of project and code which you'll find at any reasonably sized tech company, the algo-heavy code is a small fraction of the total code they write. In this sense, they are not such an outlier just based on the business they are in - I think the use of a functional language, good tooling and education around PBT are much more important factors.
If someone was already evaluating the work output using a metric closer to the underlying quality then it might not have been a big shift for them (other than having much more work to evaluate).