Container OS comparison(blog.codeship.com)
blog.codeship.com
Container OS comparison
http://blog.codeship.com/container-os-comparison/
6 comments
What about SmartOS? It's been container-native for, like, ten years and Joyent just recently added support for running Docker containers on bare metal.
I posted a reply on the original post, so I'll add it here as well:
Yes, SmartOS is indeed very interesting and has been around for many years already. They've done some incredible work to get Docker containers to work on their platform. The purpose of the comparison here is to show some of the Linux OSes that are focused on containers (mainly Docker but also others), and it's never gonna be an exhaustive list. SmartOS could've been on the list as another alternative.
Yes, SmartOS is indeed very interesting and has been around for many years already. They've done some incredible work to get Docker containers to work on their platform. The purpose of the comparison here is to show some of the Linux OSes that are focused on containers (mainly Docker but also others), and it's never gonna be an exhaustive list. SmartOS could've been on the list as another alternative.
It is quite odd running Ubuntu "GNU/Linux" not on Linux: "Linux 3.13.0 BrandZ virtual linux x86_64 x86_64 x86_64 GNU/Linux"
Seconding SmartOS. I haven't used it for running containers yet, but I've had wonderful experiences with it running Zones and traditional VMs (mostly OpenBSD, with a few Linuxen).
Triton is pretty sweet. Provisioning is a little slow, but everything is blazing fast once its running.
This focus on "container OSes" is baffling to me. Just use one of the well maintained, general purpose distros such as Debian with a minimal set of software. We don't need specialized distros for this.
The current app container model as it is takes the form of binary images managed by a de facto runtime that enforces namespace, cgroup and other boundaries by using kernel subsystems for logical partitioning.
Unlike system containers like LXC, appc means you now have an explosion of the state space whereby every container has their own independent copy of a host OS, yet relies on the semantics of its container manager to function, i.e. it is not easily introspectable with vanilla tools.
So you end up needing to build a small parallel userspace to handle things like service discovery, orchestration and provisioning, networking bridges, location transparency, resource scheduling and taken to its logical conclusion you start running a user-mode kernel (Mesos) on top of a base OS kernel (Linux) to provide further abstractions of OS resources and run extended versions of OS processes called frameworks.
I don't know if it's too early to tell, but I think people might have dug themselves into a ditch where in an attempt to escape the limitations of their kernel and userland, they've only expanded it with a concurrent layer and must maintain both.
Unlike system containers like LXC, appc means you now have an explosion of the state space whereby every container has their own independent copy of a host OS, yet relies on the semantics of its container manager to function, i.e. it is not easily introspectable with vanilla tools.
So you end up needing to build a small parallel userspace to handle things like service discovery, orchestration and provisioning, networking bridges, location transparency, resource scheduling and taken to its logical conclusion you start running a user-mode kernel (Mesos) on top of a base OS kernel (Linux) to provide further abstractions of OS resources and run extended versions of OS processes called frameworks.
I don't know if it's too early to tell, but I think people might have dug themselves into a ditch where in an attempt to escape the limitations of their kernel and userland, they've only expanded it with a concurrent layer and must maintain both.
> So you end up needing to build a small parallel userspace to handle things like service discovery, orchestration and provisioning, networking bridges, location transparency, resource scheduling
Or you install a PaaS, which does these for you. I've worked on Cloud Foundry, OpenShift is a competitor. Both are far superior to rolling your own.
Or you install a PaaS, which does these for you. I've worked on Cloud Foundry, OpenShift is a competitor. Both are far superior to rolling your own.
Yes, agreed, there are already many good OSes that can run containers fine. I run many "containers" (jails) on FreeBSD, I don't need a specialized OS for this. SmartOS can do this well. And like you said, Debian would be fine running Docker or whatever else.
I'm not so sure. General purpose distros are beholden to many interests that are likely to be irrelevant to "just" running containers, and likely to affect decision making regarding features, schedules, and priorities. One of the major advantages of Linux is that it can be customized to suit particular needs; I'm not sure why it is "baffling" to take advantage of that capability to specialize based on narrow and specific use cases.
But it's Docker so we have to rebuild everything multiple times now: six new distributions, twenty new app schedulers, fifty new network stacks. Because Docker.
Well CoreOS (for example) has a different update model, much more frequent, failover back to previous image in case of error and so on, that should help uptime overall. And it has a minimal config that can be set in a single config file. It is a different model.
It's all part of the Docker/Container cash grab I'm afraid.
There is still big amount of specialized software and things like tuned kernels, enhanced networking and so on in such OSes.
For example CoreOS comes with lots of tools helping to orchestrate and manage those clusters.
For example CoreOS comes with lots of tools helping to orchestrate and manage those clusters.
A specialized kernel package, etc. can be made for any distro.
Author here if you have any questions on the content :)
Actually, I'm very curious if you have any opinions on combining OSes and some of the tooling that's been coming out (etcd (without CoreOS), and Consul are two examples that come to my mind).
A lot of these container OS's seem to have so much built-in tooling that it seems "wrong" to consider using extra tooling on top of them.
A lot of these container OS's seem to have so much built-in tooling that it seems "wrong" to consider using extra tooling on top of them.
It's a question I've seen pop up every now and then. If you look at the current distribution landscape in any given enterprise today you'll probably see a standardization on one distribution, with a few exceptions here and there. This is mostly due to the need to have ease of management and the same goes for container OSes. If you mix several of them you're adding on "management tax" for the underlying OSes, even if the containers might be managed by a tool on top of it.
Deploying/configuring/managing CoreOS is different than RancherOS which is different than SmartOS, for instance.
I say try a few of them, including the tooling that comes with them, and since the containers look the same and will behave the same on all of them you can easily change your underlying OS (and tooling) in this testing phase. Then try the different tools that can be used on top of them and I'm sure you'll find a mix that works for you :)
I say try a few of them, including the tooling that comes with them, and since the containers look the same and will behave the same on all of them you can easily change your underlying OS (and tooling) in this testing phase. Then try the different tools that can be used on top of them and I'm sure you'll find a mix that works for you :)
I like this... it served well to get my feet wet about what is out there. It would be nice having a chart comparing the pro and cons of those OSs. Maybe in the future?
Thanks! :)
Thanks! :)
Yeah, this seemed more like a listing of the operating systems than a comparison.
You're right, it's a list of the OSes and some of their strengths. I'd rather not get into a container OS war on "which one is better", as I stated at the beginning of the article it always depends on your needs :)
Thank you! I try to list mostly the pros and not the cons, as missing features might pop up in the future which would make the post less useful.
Archlinux with systemd-nspawn is my favourite container OS. Tools like machinectl, networkctl, journalctl nicely integrated. Easy to manage multiple processes in individual containers. Streaks ahead of the competition which are seriously bloated by comparison.
smallest OS possible is no OS , second best which I use is tianon/true image from docker registry, 125 bytes, with my compiled golang app as binary image size under 5 mb, https://registry.hub.docker.com/u/tianon/true/
reference https://github.com/senthilnayagam/godockersample
edit: added a implementation link
reference https://github.com/senthilnayagam/godockersample
edit: added a implementation link
This is talking about OSs to run containers not to run in them. And why do you need true?
Apparently AWS does not support the minimal Scratch image this is based on.
You can create scratch with
tar cv --files-from /dev/null | docker import - scratch
tar cv --files-from /dev/null | docker import - scratch
Alpine seems like a glaring omission.
Alpine's a great distro, but it's not a container OS in the sense that's being defined here.