Drone and Docker, Open Source CI(blog.drone.io)
blog.drone.io
Drone and Docker, Open Source CI
http://blog.drone.io/2014/2/5/open-source-ci-docker.html
17 comments
The workflow is pretty basic right now, however, we plan on adding matrix and parallel builds in the near future. Could you elaborate a bit more on your workflow? I definitely want to make sure Drone supports more than just simple use cases.
From my experience with Jenkins, as a build/deployment/release engineer the past 6 years, you probably want to:
- chain jobs - needed for larger projects; ideally this should even allow composing jobs to have nice, modular jobs which can be launched standalone or chained
- some kind of powerful templating system - needed for reducing configuration duplication; ideally this would keep track of all the "children" in case of updates
- you also probably need enterprisey features later on, like SSO using AD/LDAP, fine grained ACLs based on groups, etc
But job chaining and job templating should be higher priorities for the workflows since they affect the overall architecture. Jenkins has been struggling for a while to re-architect to allow this, not entirely successfully.
You also want a plugin system if you don't have one, especially one with dependencies (i.e. the Git plugin can server as a dependency for the Github plugin).
My 2 € cents :)
- chain jobs - needed for larger projects; ideally this should even allow composing jobs to have nice, modular jobs which can be launched standalone or chained
- some kind of powerful templating system - needed for reducing configuration duplication; ideally this would keep track of all the "children" in case of updates
- you also probably need enterprisey features later on, like SSO using AD/LDAP, fine grained ACLs based on groups, etc
But job chaining and job templating should be higher priorities for the workflows since they affect the overall architecture. Jenkins has been struggling for a while to re-architect to allow this, not entirely successfully.
You also want a plugin system if you don't have one, especially one with dependencies (i.e. the Git plugin can server as a dependency for the Github plugin).
My 2 € cents :)
Chaining jobs and parallel ones are both very important. Especially the last one since it saves you a lot of time waiting the tests to complete. Also a big plus is to be able to run certain set of tests only when a specific event is fired eg ran test A when somebody pushes to branch X
This looks great! Looking forward to bitbucket integration.
Thanks! We posted our go-bitbucket api today as well. https://github.com/drone/go-bitbucket
The Bitbucket patch should land next week
The Bitbucket patch should land next week
So is Docker really a safe alternative to VMs? I was under the impression that you can't run untrusted code in a Docker container yet.
The open source edition is meant for teams that want to run CI on their own trusted infrastructure.
That being said, we are definitely making a long term bet on Docker and containers, and are confident security will improve over time.
That being said, we are definitely making a long term bet on Docker and containers, and are confident security will improve over time.
Why do people keep pretending this is the purpose of containers?
Why is that not the purpose of containers? FreeBSD jails have been used for security for years.
You can't run untrusted code as root in a docker container.
More specifically, you currently can't run untrusted code as root in a linux namespace, which is the default backend for Docker. There is work underway to improve the situation in 3 ways:
1) in Docker, to support backends other than lxc, including vm-mapping and openvz which have a better security track record.
2) In Linux, to further harden linux namespaces upstream so that they can safely be used to execute untrusted code as root [1]
3) in ops best practices, to combine linux namespaces with additional security measures (selinux, apparmor, clustering to deploy mutually untrusted containers on different docker hosts, etc).
[1] a big focus of the namespacing effort us user namespaces which makes a container "think" it runs as root when in fact it doesn't. User namespaces work great but haven't been around long enough to be vetted. Beyond that, namespaces are pretty robust and feature-complete already. What's left is to go through the process of auditing, testing and generally allowing it to stand the test of time and scrutiny. Eventually ops and security engineers will warm up to it and it will graduate to "production-ready", the way Zones, Jails and OpenVZ did before it. It's only a matter of time.
1) in Docker, to support backends other than lxc, including vm-mapping and openvz which have a better security track record.
2) In Linux, to further harden linux namespaces upstream so that they can safely be used to execute untrusted code as root [1]
3) in ops best practices, to combine linux namespaces with additional security measures (selinux, apparmor, clustering to deploy mutually untrusted containers on different docker hosts, etc).
[1] a big focus of the namespacing effort us user namespaces which makes a container "think" it runs as root when in fact it doesn't. User namespaces work great but haven't been around long enough to be vetted. Beyond that, namespaces are pretty robust and feature-complete already. What's left is to go through the process of auditing, testing and generally allowing it to stand the test of time and scrutiny. Eventually ops and security engineers will warm up to it and it will graduate to "production-ready", the way Zones, Jails and OpenVZ did before it. It's only a matter of time.
Very excited about the Docker integration!
I saw a demo of this a week ago, and it looked really neat. Kudos for launching!
I had the chance of seeing a demo of this yesterday; it went really, really great. Brad is a great guy with a ton of passion.
Drone combined with some of the features upcoming on docker.io will be an incredibly compelling usecase for Continuous Integration going forward.
Drone combined with some of the features upcoming on docker.io will be an incredibly compelling usecase for Continuous Integration going forward.
Wow just yesterday I was visualizing and dreaming about what a containerized CI system might look like after realizing:
1) I don't want to pay for hosted CI
2) Setting up your own CI is a pain in the butt currently
Well done. I'm gonna have a look through this!
1) I don't want to pay for hosted CI
2) Setting up your own CI is a pain in the butt currently
Well done. I'm gonna have a look through this!
Here here! I saw a demo as well and what he has built as well as what he has planned on the roadmap is great. Love where its going.
Love seeing an alternative for jenkins, it's about time!
I haven't really looked closely at Drone yet, but you might also be interested in Strider. http://stridercd.com/
One of the things I like is that it's dead simple to get running on Heroku. Language support is a little weak (Python, Ruby, node.js), but we're working on that.
Another nice feature that's currently lacking which it looks like Drone does well is the ability to provision external services (e.g. DB servers) for tests.
One of the things I like is that it's dead simple to get running on Heroku. Language support is a little weak (Python, Ruby, node.js), but we're working on that.
Another nice feature that's currently lacking which it looks like Drone does well is the ability to provision external services (e.g. DB servers) for tests.
So I can run this locally, ensure my test system is dialed in, then scale/automate it with the hosted service? If that's true, it certainly beats having to guess what my test system is actually doing remotely and would definitely help bring some alignment between my prod/test infrastructure.
yes! there is a CLI that let's your run your builds locally, on your laptop. Navigate to the root directory of your repository and run `drone -v build .`
you need Docker installed and the .drone.yml file in the root. it's a great way to test locally without having to push to the CI server. As an added bonus, you could even setup a pre-commit hook
you need Docker installed and the .drone.yml file in the root. it's a great way to test locally without having to push to the CI server. As an added bonus, you could even setup a pre-commit hook
[deleted](1)
Extremely well done Brad. With Drone, we can finally ditch Jenkins. Going to setup Drone open source for https://commando.io right now!
There's an article on VentureBeat posted today with some more info about Drone.io: http://venturebeat.com/2014/02/07/droneio
Having seen a demo with Brad @ Geekdom SF. Drone.io is to jenkins what IntelliJ is to Eclipse.
What I see as a benefit is how all the features of this CI system work for you out of the box. Definitely worth a look.
What I see as a benefit is how all the features of this CI system work for you out of the box. Definitely worth a look.
so I wonder how this impacts the circleCI funding. Looks like some pretty steep competition. http://blog.circleci.com/we-raised-a-6m-series-a/
This is exactly what I've been looking for, docker + CI, and I'm excited to see the scala 2.10 image!
Great work from a great Geekdom startup! Yay!
Does it provide the fine-grained workflows Jenkins does?