We've been working quite a bit on bringing in Nix/Nixpkg/Nixos based architecture into the space of bringing in determinism to the baseline infrastructure of our AI stacks. This also leans into the conversation around sovereign AI and how Nix is letting us own more of that stack.
Asking this with a bias since I work on Nixos.org and Flox.dev - How is the team thinking about the infra layers underneath these models? Any priority or reason to imbed determinism/reproducibility at the bottom of the stack?
What's the best practice right now for setting these up? We've been primarily using Nix/Flox to set up the models pretty quickly and at least with minimized amount of commands(biased Nix/Floxer) here and found it useful
A small note of amazement from Nix/Flox person here. Incredible to see this release and congrats! Mike, you're an allstar for so many years of contributions!
Ron from Flox here. We recently wrote this piece on how Nix can address the emerging CVE remediation problem. We are curious to hear any feedback on our approach.
There's a lot of reasons to use Nix instead of or WITH Homebrew depending on your exact needs.
Where it’s paid off for me (and where I think it actually wins) is when the problem is recreating environments: multiple machines, teammates, CI, nasty native deps, CUDA stacks, etc. At that point you’re choosing where entropy lives: in invisible drift (brew/manual installs) or in a repo you can diff/rollback.
Also, you don’t always need to go full “immutable everything.” Really depends on your needs here. Hybrid tends to be another sane path. In certain situations this can get you 80% of the upside without having to rip it all out. So kinda the "good enough" which I've seen a lot of folks do.
Highly recommend to check this out, the blog/Arnoult does an amazing job in very succinctly breaking down the aspects of SBOMs in a Nix based infra approach. We can go way beyond the current SLSA levels and provide full provenance at the atomic level of the supply chain for when it's needed. And as Arnoult points out, prune when it's not.
There's good work being done on this across the Nix ecosystem and we have also seen a lot of use for it come in through Flox as well!
Jotting down a few quick thoughts here but we can totally go deep.
This is something Michael Brantley started working on a few months ago to test out how to make it super easy to ease and leverage existing Nix & Flox architecture.
One of the core differences from my quick perspective is that it specifically leverages the unique way that Flox environments are rendered without performing a nix evaluation, making it safe and optimally performant for the k8s node to realize the packages directly on the node, outside of a container.
Yes, this hits the nail on the head. We’ve seen the same explosion in image size and rebuild complexity, especially with AI/ML workloads where Python + CUDA + random pip wheels + system libs = image bloat and massive rebuilds.
With the Kubernetes shim, you can run the hash-pinned environments without building or pulling an image at all. It starts the pod with a stub, then activates the exact runtime from a node-local store.
Going to sound weird but with both my hats on I super appreciate this perspective. I can only speak to some areas of Nix and Flox obviously and I know folks are looking into doing this to your point a whole lot better. Zooming in way more into solving for us that just want to run and fix it fast when it breaks.
Also, think it's a huge ecosystem win for FreeBSD pushing on reproducibility too. I think we are trending in a direction where this just becomes a critical principle for certain stacks. (also needed when you dive into AI stacks/infra...)
Ron from Flox here, woke up to feed a brand new 3 day old to see this here! On about 3 hours of sleep (over the lat 48 hours) but excited to try and answer some questions! Feel free to also drop any below <3
+1 to Farid, great write-up! What you’re seeing is the long-standing “deriver” mismatch: fixed-output derivations can change their .drv without changing the output path. Eelco is calling it out as well in the comment below. I believe the idea behind the path forward is there but happy to hear more!
I've run into similar pain points with GitHub Actions in past roles but I still very much use them/get value. One approach that's helped us at Flox is indeed using Nix and we've now seen customers start leveraging that. Significant drop in “works on my machine” issues and more reliable CI pipelines, etc... It’s not a silver bullet, but it offers a solid technical foundation for tackling these challenges.
On the Flox side we are very much on the integrate and improve rather than full replace for scenarios like this one <3
Happy to answer any Nix items on this!
Awesome to see this on here! We've seen a lot of comments talk about Nix versus other tools recently and wanted to share some of our thoughts on it. We like seeing Nix meet developers where they are, integrating into our existing workflows, including containerized ones.