the FreeBSD migration didn’t take that long - iirc this was the frontend migration to an IIS ISAPI.
the Solaris bits (storage and routing tables) took far longer - and again iirc the frontend had been rewritten in C# before all the Sun hardware had been decommissioned.
you pull packages from a trusted package repository, not from the internet. this is not rare in my experience (financial services, security) and will become increasingly common due to software supply chain issues.
having a local simulator (DynamoDB, Spanner, others) helps me a lot for offline/local development and CI. when a vendor doesn't off this I have often end up mocking it out (one way or another) and have to wait for integration or e2e tests for feedback that could have been pushed further to the left.
in many CI environments unit tests don't have network access, it's not purely a price consideration.
(not a turbopuffer customer but I have been looking at it)
Misleading headline, the first sentence is "how Spotify optimized and sped up elements from *our largest Dataflow job*". Surely it's not the largest ever run, even on Dataflow.
Hive also has had this optimization for as long as I can remember. As others have noted it's not particularly new or novel, it's just not part of the Beam SDK.
I wonder why the interested parties don't just offer to pay for ID cards. Seems like this would solve both problems, but perhaps I'm missing something obvious.
fun times for sure.