I job hop a bunch and a common theme that I have had in my jobs is to go in and understand the legacy, and bring it up to a more reasonable state. It's been interesting having to be the one dealing with someone else's consequences.
I'll sometimes chat with the people who have had to maintain whatever ground work I had laid, and each time around, I get a little bit better at laying down a framework of infrastructure and operations that's better and better at staying sustainable
I had the pleasure to briefly work with the author of this post within the last few years. Jeff was one of the most enlightening and positive people I've ever learned from. He was refreshingly honest about what challenges he was having, and delightfully accessible for mentorship and advice.
Depot also does remote docker builds using a remote build kit agent. It was actually their original product. If you could feasibly put everything into a Dockerfile, including running your tests, then you could use that product and get the benefits.
>It’s simple to create a Dockerfile that containerizes your app. It’s even easier to create one that has terrible build performance, chalked full of CVEs, is north of 20GB in size, and whatever foot gun you trip over when using Docker
It's been years that writing Dockerfiles has been fairly common. Years. And yet it's still so common that people write such poorly optimized Dockerfiles.
I think we should definitely start thinking about admitting that it's too much over head for people to learn how to write a Dockerfile.
That being said, I've known Kyle for a while now. The team at Depot have consistently shown the deepest possible understanding of the container ecosystem. I'm very excited to see what else they do.
As opposed to keeping all of your servers independent of each other, super computers are used any time you want to pretend the entire computer is one computer.
In other words, they're used when you want to share some kind of state across all of the computers, without the potential overhead of communicating to some other system like a database.
Physics simulations and like, molecular modeling come to mind as common examples.
In the case of ML training, model parameters and broadcasting the deltas that get calculated during training are that shared state.
In my experience, running a plan is much less likely to catch a bad value than the AWS provider.
Subjectively, the AWS provider will at least validate that fields have valid values during the plan step. The Google provider doesn't seem to validate actual values until apply, and then you get a failure
Search is the majority of their revenue so that they can subsidize their other products. Devastation somehow doesn't feel like it properly captures how much it would hurt them to lose it.
I used to open source every single thing I built. But recently, I stopped doing that because now I build everything as a single monorepo. All of my completely disconnected side project ideas are all in one repo and I'm much more happy being able to keep this like dependencies up to date in one place instead of having a different repo per project.
I would like to go back to open sourcing all things I play with but maybe I need to find a repo structure and tooling that would make that more sustainable.
The highlight of the fact that large org adoption is slow feels really valuable here. Especially when we just recently had all of the articles that claimed that ChatGPT was "over" when it appears that this is mostly because of things like summer vacations with students. The adoption and understanding of these products and the risk involved with hallucinations is something that will take time to understand.
I recently joined one of the LLM provider companies and watching these phases over the next few years will be really interesting. Especially combined with what's going on with regulation and the like.
Random aside, hi Elad! I think you're reading some of these comments. I just left Color after ~2.5 years. I hope to get to formally introduce myself to you one day.