A rant about NixOS(dev-ops-notes.com)
dev-ops-notes.com
A rant about NixOS
https://dev-ops-notes.com/nixos/why-you-should-never-ever-use-nixos/
9 comments
> 100% CPU utilizations will require AutoScaling Group to launch additional instances
If you know your instance takes long to build, you should set the health check grace period. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
Or if the checked service comes up before the system is ready, set a cooldown on the scaling up https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
Or in extreme cases, prepare a complete AMI to boot from. This is not a NixOS issue. This it completely on how you operate your AWS.
If you know your instance takes long to build, you should set the health check grace period. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
Or if the checked service comes up before the system is ready, set a cooldown on the scaling up https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
Or in extreme cases, prepare a complete AMI to boot from. This is not a NixOS issue. This it completely on how you operate your AWS.
This doesn't seem very convincing. Disclosure: I run NixOS on all of my machines. Even my router.
Point by point:
> Small community
This reason applies to any new tool that doesn't immediately have significant corporate backing. Nothing would ever grow organically if everyone thought this was a deal-breaker. The real point here is that if you use NixOS then you must be committed to be an early adopter and bear all the costs that entails. If you don't get high value out of Nix, then you shouldn't adopt it. Simple enough, yea?
> SOFTWARE MANAGEMENT
This section seems a bit completely ridiculous? Yes, you can't configure your NixOS system unless you write nix expressions. That's literally the point of using NixOS. You don't boot up emacs and complain that it doesn't care about your .vimrc.
> Want to customize something in the OS, be ready, you’ll need to write your custom systemd service to do that. Aaand, yes, service configuration file also need to be declared using Nix expression language.
Yes. That's the point. As much of your system configuration is configured in a principled way.
> All standard system configuration files like /etc/fstab, for example are generated by a weird code and read only.
Yes. That's the point. You can only modify your system configuration in a principled way.
> Forget about Chef, Puppet or Ansible! Only Nix expressions! Only hardcore!
Yes. That's the point. That's what you sign up for when you install NixOS.
> It is not possible just to upgrade the kernel from “ver1” to “ver2”. New kernel will bring whole set of system packages and their dependencies with it. Do not know if it is safe. Will test it soon.
Wait, does this person think you can ad-hoc change your kernel version without considering whether your low level and/or kernel specific packages are compatible with it? That's _the point_ of system packages being parameterized on kernel version in NixOS.
> CLOUD SUPPORT
Yeah, actually you have to be knowledgeable in both Nix and whatever cloud service you're using to get by without issue. It's an early adopter cost, as mentioned before.
> Do not even try to use it in Auto Scaling group and passing configuration.nix content through user-data. ...
I'm pretty sure the author was not deploying NixOS to the cloud correctly, based on this comment. But! It's not obvious how to do it correctly, so understandable.
> Cache
I don't even understand the issue here? You don't have to rely on the community cache. Our shop builds the entirety of nixpkgs and caches it for ourselves as a matter of routine.
> And yes, your own cache size will also grow very fast!
Well yes. You have a cache that contains every artifact relevant to your business going back as far as you want. That sounds rad, actually? Again. That's the point.
> SECURITY
Yea, that sucks. I also dislike when the community introduces breaking changes. This is yet another early adopter cost and if you're not involved in the community the cost can be shocking and difficult to deal with. I sympathize with the author.
> WINDOWS SUPPORT
This Linux distribution doesn't support... Windows? Ok, I've never worked at a Windows shop. What sort of support for Windows do other Linux distributions have that the author is lacking in? It is very true that if you need to support Windows you shouldn't use NixOS.
> SYSTEM REQUIREMENTS
That's what the cache is for my guy :^)
> Conclusion
I'm really sorry for the author, because it looks like somebody made a rash decision to use NixOS that did not get everyone on board and did not fully commit to leveraging NixOS's advantages to outweigh its high early adopter costs.
But I think this rant is more than a little misinformed.
Point by point:
> Small community
This reason applies to any new tool that doesn't immediately have significant corporate backing. Nothing would ever grow organically if everyone thought this was a deal-breaker. The real point here is that if you use NixOS then you must be committed to be an early adopter and bear all the costs that entails. If you don't get high value out of Nix, then you shouldn't adopt it. Simple enough, yea?
> SOFTWARE MANAGEMENT
This section seems a bit completely ridiculous? Yes, you can't configure your NixOS system unless you write nix expressions. That's literally the point of using NixOS. You don't boot up emacs and complain that it doesn't care about your .vimrc.
> Want to customize something in the OS, be ready, you’ll need to write your custom systemd service to do that. Aaand, yes, service configuration file also need to be declared using Nix expression language.
Yes. That's the point. As much of your system configuration is configured in a principled way.
> All standard system configuration files like /etc/fstab, for example are generated by a weird code and read only.
Yes. That's the point. You can only modify your system configuration in a principled way.
> Forget about Chef, Puppet or Ansible! Only Nix expressions! Only hardcore!
Yes. That's the point. That's what you sign up for when you install NixOS.
> It is not possible just to upgrade the kernel from “ver1” to “ver2”. New kernel will bring whole set of system packages and their dependencies with it. Do not know if it is safe. Will test it soon.
Wait, does this person think you can ad-hoc change your kernel version without considering whether your low level and/or kernel specific packages are compatible with it? That's _the point_ of system packages being parameterized on kernel version in NixOS.
> CLOUD SUPPORT
Yeah, actually you have to be knowledgeable in both Nix and whatever cloud service you're using to get by without issue. It's an early adopter cost, as mentioned before.
> Do not even try to use it in Auto Scaling group and passing configuration.nix content through user-data. ...
I'm pretty sure the author was not deploying NixOS to the cloud correctly, based on this comment. But! It's not obvious how to do it correctly, so understandable.
> Cache
I don't even understand the issue here? You don't have to rely on the community cache. Our shop builds the entirety of nixpkgs and caches it for ourselves as a matter of routine.
> And yes, your own cache size will also grow very fast!
Well yes. You have a cache that contains every artifact relevant to your business going back as far as you want. That sounds rad, actually? Again. That's the point.
> SECURITY
Yea, that sucks. I also dislike when the community introduces breaking changes. This is yet another early adopter cost and if you're not involved in the community the cost can be shocking and difficult to deal with. I sympathize with the author.
> WINDOWS SUPPORT
This Linux distribution doesn't support... Windows? Ok, I've never worked at a Windows shop. What sort of support for Windows do other Linux distributions have that the author is lacking in? It is very true that if you need to support Windows you shouldn't use NixOS.
> SYSTEM REQUIREMENTS
That's what the cache is for my guy :^)
> Conclusion
I'm really sorry for the author, because it looks like somebody made a rash decision to use NixOS that did not get everyone on board and did not fully commit to leveraging NixOS's advantages to outweigh its high early adopter costs.
But I think this rant is more than a little misinformed.
I agree with the vast majority of your post, but..
> Wait, does this person think you can ad-hoc change your kernel version without considering whether your low level and/or kernel specific packages are compatible with it? That's _the point_ of system packages being parameterized on kernel version in NixOS.
Linux has done an amazing job of clearly defining the kernel and userland boundaries while fanatically maintaining (recent) backwards compatibility from a new kernel to an older userland. The kernel and the modules/headers associated with the kernel should be treated the same as any other package, and arguably almost no other package on the system should have a hard dependency against a kernel package. It's entirely reasonable to wish to build a rootfs/userland and ship it to be run under an arbitrary linux kernel you dont control. (i.e. paravirt vms, container images, and devices only supported by a vendor kernel).
> Wait, does this person think you can ad-hoc change your kernel version without considering whether your low level and/or kernel specific packages are compatible with it? That's _the point_ of system packages being parameterized on kernel version in NixOS.
Linux has done an amazing job of clearly defining the kernel and userland boundaries while fanatically maintaining (recent) backwards compatibility from a new kernel to an older userland. The kernel and the modules/headers associated with the kernel should be treated the same as any other package, and arguably almost no other package on the system should have a hard dependency against a kernel package. It's entirely reasonable to wish to build a rootfs/userland and ship it to be run under an arbitrary linux kernel you dont control. (i.e. paravirt vms, container images, and devices only supported by a vendor kernel).
That's entirely reasonable, and if you're in such a situation you probably don't want to ship a NixOS image. I've been in situations like that and I still prefer to use Nix (which is just the dependency manager) to build such images. I'd be curious if you could take a look and peruse this list of packages that are Kernel dependent and let me know which you think are unreasonably dependent: https://pastebin.com/njS40UzP
If you operate from the mindset that nix is always responsible for all of the kernel and all of userland, none are unreasonable.
The moment you try to live in a world where nix isn't controlling the kernel, most of those packages are unreasonable. Several provide both a kernel module and userspace tools. Wireguard, VirtualBox, OpenAFS are prime examples that jump out at me. It's totally reasonable to wish to install the userland portion of any of these packages without adding a kernel you cant even use to your system image.
My personal preference would be to see such packages split, with the kernel module package treated as an optional dependency for the userland package. Then the kmod can drag in the kernel and headers only when needed.
The moment you try to live in a world where nix isn't controlling the kernel, most of those packages are unreasonable. Several provide both a kernel module and userspace tools. Wireguard, VirtualBox, OpenAFS are prime examples that jump out at me. It's totally reasonable to wish to install the userland portion of any of these packages without adding a kernel you cant even use to your system image.
My personal preference would be to see such packages split, with the kernel module package treated as an optional dependency for the userland package. Then the kmod can drag in the kernel and headers only when needed.
Fortunately, the userland functionalities of those examples are packaged separately, I checked just to be sure:
wireguard: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/netw...
virtualbox: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio...
openafs: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/op...
Of course, this is not well documented so I understand that someone who isn't knowledgeable about NixOS would presume like the author that they had to depend on a Linux kernel to get these userland packages. Even I can't say with 100% confidence that everything packaged up under linuxPackages has a kernel agnostic version derivation for its userland components.
I remember being very skeptical of NixOS when I first started using it and it seemed like it was trying its best to prove me right. It really was my dislike for Ubuntu that motivated me to go all the way.
wireguard: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/netw...
virtualbox: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio...
openafs: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/op...
Of course, this is not well documented so I understand that someone who isn't knowledgeable about NixOS would presume like the author that they had to depend on a Linux kernel to get these userland packages. Even I can't say with 100% confidence that everything packaged up under linuxPackages has a kernel agnostic version derivation for its userland components.
I remember being very skeptical of NixOS when I first started using it and it seemed like it was trying its best to prove me right. It really was my dislike for Ubuntu that motivated me to go all the way.
Is passing configuration through userdata considered bad practice?
The most common practice I've seen is to build a configured system on another machine and ship it to EC2 via `nix copy` or `nix-copy-closure` and then activate it by running `switch-to-configuration`.
Nope, not at all. Dockerfiles are not deterministic. Most of the ways I see Dockerfiles written are highly non-deterministic. Docker also requires root access and to some degree exposes that root access to any user with Docker access on a host.
The "stacked filesystem" model also allows far less flexibility than Nix does when it comes to composing a system out of different components. The fundamental problem is the inability to combine features of Docker images with different bases. Compare this with OO languages lacking multiple inheritance.
However, in my eyes, the biggest sin of Docker is the conflation of the problem of software dependency management with isolation & security policy. The two are fundamentally different things, causing developers pain when doing local development and leading to security-slackening changes that make their way into production. NixOS does not make this mistake.
Last time I was running a project with NixOS I was able to git bisect my entire application & software stack to locate the source of a regression (a package maintainer had changed the location of a file) - what can you do?