Ask HN: Which tool(s) do you use to install/upgrade your app on production?
Ask HN: Which tool(s) do you use to install/upgrade your app on production?
14 comments
We use puppet for configuration that varies between systems, but otherwise bundle our software as debian packages so just need to run 'apt-get dist-upgrade' per machine. Separate private repositories for testing/staging/production sets of the packages.
We use puppet for managing packages and configuration on our systems. At some point I'd like to take a look at Chef, too -- having Ruby as the configuration language seems like it'd be nicer than puppet's config file format. It took some work to get set up, but it's so nice being able to keep track of what's on each machine in version controlled source files.
For actually deploying our code, we use Capistrano, which we also use to run one-off tasks across machines. This pulls from our code repsitory, and each of our dev/staging/production environments know which version control branch to pull and deploy in that environment.
For actually deploying our code, we use Capistrano, which we also use to run one-off tasks across machines. This pulls from our code repsitory, and each of our dev/staging/production environments know which version control branch to pull and deploy in that environment.
What do you use to produce dpkgs? In addition to the cumbersome standard tools, I'm only aware of jpkg (http://code.google.com/p/jpkg-library/) for Java.
I've found the dpkg method to be very effective. You can sign the packages, mirror the repositories, and set up servers to automatically upgrade some/all of the packages via cron jobs.
Packages can be automatically generated by your continual integration system, which also has the nice effect of creating a fully-trusted and centralized deployment path, making it difficult to sneak in uncommitted changes and non-managed files, etc.
The packages themselves support complex dependencies, start/stop scripts, etc, and dpkg/apt-get themselves have been ported to non-Linux platforms.
Personally, I'd still like a more comprehensive centrally managed solution based around the packaging model (ie, each server runs a management agent that can install/remove/update packages, etc).
I've found the dpkg method to be very effective. You can sign the packages, mirror the repositories, and set up servers to automatically upgrade some/all of the packages via cron jobs.
Packages can be automatically generated by your continual integration system, which also has the nice effect of creating a fully-trusted and centralized deployment path, making it difficult to sneak in uncommitted changes and non-managed files, etc.
The packages themselves support complex dependencies, start/stop scripts, etc, and dpkg/apt-get themselves have been ported to non-Linux platforms.
Personally, I'd still like a more comprehensive centrally managed solution based around the packaging model (ie, each server runs a management agent that can install/remove/update packages, etc).
I am just finishing up something akin to what you described in your last line (for my employer). It's just a simple daemon written in Python that is told what to do via XML/HTTP. Implementing such a daemon was made much easier by using the python-apt module.
Any chance of open sourcing it? It would be pretty cool to couple that with a central, authenticated server management web interface.
We use an absolute minimum in terms of packaging (#) - after all, we don't need to don't need to go out of our way to make it easy to re-build. Packages are mostly C/C++ and python apps, so fortunately none of the extra issues with Java packaging.
Oh, and dependencies are great.
(#) Link has gone now, but available at http://web.archive.org/web/20060925094756/http://www.kclee.d...
Oh, and dependencies are great.
(#) Link has gone now, but available at http://web.archive.org/web/20060925094756/http://www.kclee.d...
I use capistrano and in some cases i use debian packages. As the debian tools don't work on OS X. I've made a debian package tool called apt_fu that can run on any unix system. It's at github:
http://github.com/sunkencity/apt_fu/tree/master
it's really a quick hack, but it get's the job done.
http://github.com/sunkencity/apt_fu/tree/master
it's really a quick hack, but it get's the job done.
You can install apt-get and dpkg on Mac OS X via MacPorts (but apt_fu looks quite a bit nicer than using the dpkg tools).
[deleted]
capistrano.
You should spend time understanding all of the given capistrano recipes before you should start building any of your own since many things are already done for you.
I wasn't able to do this without staring at the capistrano code for a significant amount of time, but it was worth the effort IMHO.
You should spend time understanding all of the given capistrano recipes before you should start building any of your own since many things are already done for you.
I wasn't able to do this without staring at the capistrano code for a significant amount of time, but it was worth the effort IMHO.
Mercurial (distributed revision control system) --
$ hg pull ; hg updatewe too use this. we have a waterfall build tool called Merx (custom in house, hopefully soon OS) that makes nightly builds from out dev and stable hg repositories.
Thhe builds themselves go into a package repository and on a machine we just run hg pull packages/<tool>; hg merx <machine name>; hg update;
(hg merx pulls customised machine configuration files from the repository or defaults if non-existant).
It takes a bit of initial setup but then works well for ongoing deployment (we add 10-15 commodity servers to our pool each week = easy to do now - dd the debian image and run those three commands. sorted).
Thhe builds themselves go into a package repository and on a machine we just run hg pull packages/<tool>; hg merx <machine name>; hg update;
(hg merx pulls customised machine configuration files from the repository or defaults if non-existant).
It takes a bit of initial setup but then works well for ongoing deployment (we add 10-15 commodity servers to our pool each week = easy to do now - dd the debian image and run those three commands. sorted).
Lately I've been using Fabric: http://docs.fabfile.org/
So far it meets my needs well. The only issue I had was getting my SSH key working, but I was eventually able to iron that out.
So far it meets my needs well. The only issue I had was getting my SSH key working, but I was eventually able to iron that out.
I recently setup a Rails app to deploy to EC2 with chef and chef-deploy (http://github.com/ezmobius/chef-deploy).
I think chef has a lot of potential. However, I found the learning curve rather high. If you end up using it, you'll definitely want to use a VM with snapshots.
I think chef has a lot of potential. However, I found the learning curve rather high. If you end up using it, you'll definitely want to use a VM with snapshots.
I use Git in with a detached working copy a la http://toroid.org/ams/git-website-howto but with a little more flair for schema updates and things like that.
So my deployment looks like:
So my deployment looks like:
git push liveFabric: http://docs.fabfile.org/
I use fabric too, it's easy because most of the developments done here are in Python and writing a "deployment script" in the same language makes sense. The only problem I get with this is the "socket closed" error from time to time.
We don't use source control on our production machines (which I think is crap), so I use an rsync script from my dev machine. Many people just scp the files over every time they make a change. (There was once a tendency by a few folks to directly edit files on production, without ever checking these changes in to SVN; thank god this has decreased over time.) Work is also too cheap to buy SQL migration tools for everyone who needs them, so database changes are applied by hand. Not awesome.
darcs push
ssh remote darcs --repo=live pull ../test
make release
Deploy? We use Capistrano which is ok for simple setups, but it gets hairy if you have multiple configurations (staging, production).
capistrano-ext multistage
http://weblog.jamisbuck.org/2007/7/23/capistrano-multistage
http://weblog.jamisbuck.org/2007/7/23/capistrano-multistage
git push && ssh wherever 'cd /wherever; git pull; [touch or restart something]'I tend to "git pull --rebase", so that local config changes "stay on top" and don't become a merge commit.
In general, I recommend against "git pull" unless you are intentionally trying to merge something. (Fast-forwards are a special case of merging.)
In general, I recommend against "git pull" unless you are intentionally trying to merge something. (Fast-forwards are a special case of merging.)
Good point, although I maintain different environment configurations separate from the code base (or .gitignore anything that doesn't fall into that).
I wrote my own custom python program that pretty much acts like a distributed shell to automate pushes (svn/git,etc)
Its pretty close to this: http://www.netfort.gr.jp/~dancer/software/dsh.html.en
Its pretty close to this: http://www.netfort.gr.jp/~dancer/software/dsh.html.en
Puppet with RPMs in yum repositories. Python scripts push individual RPMs to the development repository, then to staging, and finally to production.
Also a few custom tweaks to restart the Puppet daemons when they die, which they do frequently.
Also a few custom tweaks to restart the Puppet daemons when they die, which they do frequently.
I use git on development machines and have an rsync script to push updates to the server.