Packer is a tool for creating identical machine images for multiple platforms(packer.io)
packer.io
Packer is a tool for creating identical machine images for multiple platforms
http://www.packer.io/
19 comments
A tutorial showing every step from start to finish for creating something simple, such as a webserver running a database with some custom html pages would be really great.
The docs look good, but it's hard to see which steps to follow in what order for someone who hasn't tried to automate this kind of thing before.
The docs look good, but it's hard to see which steps to follow in what order for someone who hasn't tried to automate this kind of thing before.
The intro actually has a decent step-by-step getting started guide: http://www.packer.io/intro I welcome any feedback!
When you're configuring the image, is the network stack usable? I currently use Boxgrinder, and the most frustrating limitation is that I can't start and interact with network daemons to initialize things that way, because it's really just a chroot jail (in the Linux sense). Consequently, I have to isolate the embedded databases of the services I'm configuring and install them directly as files.
Yes, everything is usable.
This looks great, nice work.
For the dev/prod parity use case, where do you usually draw the line between Packer and Vagrant? I'm currently building boxes with Veewee with all dependencies pre-installed and distributing this to the team. Vagrant could easily deal with this, installing dependencies when the VM is launched using provisioners.
Does anyone have any preferences/reasons to go one way or another?
For the dev/prod parity use case, where do you usually draw the line between Packer and Vagrant? I'm currently building boxes with Veewee with all dependencies pre-installed and distributing this to the team. Vagrant could easily deal with this, installing dependencies when the VM is launched using provisioners.
Does anyone have any preferences/reasons to go one way or another?
It depends on what works best for you. In big organizations, I prefer to take the Veewee approach (although in this case replacing Veewee with Packer). If you use Veewee, see: http://www.packer.io/docs/templates/veewee-to-packer.html
On your high level descriptions of your tools, like packer, and vagrant, when you say "VMware" you really need to start being clear about which products you specifically support.
It is intentionally vague as I intend to support all of them. I apologize for the confusion, however.
Great stuff! I hope this means we'll see a bunch of identical prepackaged Vagrant boxes for all these platforms soon.
Yes, yes you will. The Vagrant boxes are now being built with Packer (but not uploaded yet).
How does packer compare to boxgrinder?
Okay, can someone please explain Packer/Vagrant/etc to me? The way I currently provision is using ansible, I have a script that goes from bare machine to serving the site in one command (http://www.stavros.io/posts/example-provisioning-and-deploym...).
How does something like Packer/Vagrant help me? From what I understand, all they do is create a VM, which just takes two seconds in VirtualBox anyway. There must be a benefit, since everyone is using them, but I'm missing it. Can anyone please explain?
How does something like Packer/Vagrant help me? From what I understand, all they do is create a VM, which just takes two seconds in VirtualBox anyway. There must be a benefit, since everyone is using them, but I'm missing it. Can anyone please explain?
> How does something like Packer/Vagrant help me?
Totally depends on what you need doing. Might want to check out use cases in the packer documentation.
There is some overlap in all these tools. ThAnsible/Salt/Chef/Puppet tend to used to manage "live" OS's/systems, keeping them updated, deploying new builds to them. Although some like Ansible are more generic.
Vagrant and Packer(I know little) manage images. Starting, stoping, provisioning (and often running one of the above Configuration Managments to finish customizing. Although some believe in running from pure AMI, updating them instead of the live systems).
They get more power when there are many devs, lots of images. I.E. when you have enough complexity you need help managing it.
If your small or simple (no negative connotation implied) they probably aren't worth it. I'm one man shop and use Ansible cause it's the right fit for that (although it's expanding fast).
Totally depends on what you need doing. Might want to check out use cases in the packer documentation.
There is some overlap in all these tools. ThAnsible/Salt/Chef/Puppet tend to used to manage "live" OS's/systems, keeping them updated, deploying new builds to them. Although some like Ansible are more generic.
Vagrant and Packer(I know little) manage images. Starting, stoping, provisioning (and often running one of the above Configuration Managments to finish customizing. Although some believe in running from pure AMI, updating them instead of the live systems).
They get more power when there are many devs, lots of images. I.E. when you have enough complexity you need help managing it.
If your small or simple (no negative connotation implied) they probably aren't worth it. I'm one man shop and use Ansible cause it's the right fit for that (although it's expanding fast).
Hmm, so what they do is generate an image, "freeze" and deploy it? I'm not very big on that philosophy, I prefer having a script to take the system from a clean state to the current state at all times, but I guess that's just preference.
Maybe it's just that I don't have a big enough need for it, but I did see people using it for single VMs/servers, so I was wondering.
Thanks for the explanation! Also, have you seen ansible's pull mode? I just saw it the other day and I am itching to try it, it looks phenomenal.
Maybe it's just that I don't have a big enough need for it, but I did see people using it for single VMs/servers, so I was wondering.
Thanks for the explanation! Also, have you seen ansible's pull mode? I just saw it the other day and I am itching to try it, it looks phenomenal.
Well Vagrant existed as a way to take some known state and run it headless. It was good to say, "Here is the image we run on, now develop on it", where that image may not be your actual desktop machine. It helps avoid the "issue" of the problem being yours (desktop, rbenv, wrong openssl etc) and not mine (or code). This worked by running Virtual box in a headless mode and doing some nifty things to the VM (through SSH) for you.
Then it took on the optional use case of also running some Provisioning ontop of the VM. We could start from a base ubuntu, get to a point where our app could run and then deploy the app ontop. It was a little cumbersome but it worked.
Then Mitchel started adding other platforms for it to run under (VMWare Workstation) instead of just VirtualBox. I believe he is still working on these and is adding more like VMWare VSphere.
So Vagrant was always more to do with running of the VMS or taking the VMs and putting them somewhere and turning them on. Packer is more of the former tool of creating the VM in the first place I guess.
I feel from the little I do know that Packer and Vagrant seem to be sharing some of the same goals and solutions. I guess mitchel and comment further on where he plans to draw the line between the two.
Then it took on the optional use case of also running some Provisioning ontop of the VM. We could start from a base ubuntu, get to a point where our app could run and then deploy the app ontop. It was a little cumbersome but it worked.
Then Mitchel started adding other platforms for it to run under (VMWare Workstation) instead of just VirtualBox. I believe he is still working on these and is adding more like VMWare VSphere.
So Vagrant was always more to do with running of the VMS or taking the VMs and putting them somewhere and turning them on. Packer is more of the former tool of creating the VM in the first place I guess.
I feel from the little I do know that Packer and Vagrant seem to be sharing some of the same goals and solutions. I guess mitchel and comment further on where he plans to draw the line between the two.
I see, thanks. I guess creating VMs has never been a big problem for me, but, then again, it sounds like Vagrant isn't targeted towards me.
I'm not familiar with ansible so I won't attempt to compare the two but I can offer you a short list of benefits that Vagrant provides over building VMs manually with Virtualbox, VMWare, etc.
1. Single command setup across virtualization platforms
2. Tight integration with provisioning tools (ships local config to VM and runs)
3. Network configuration*
4. Automated shared folders (includes NFS since VBox has really bad shared folders)
5. Generally homogeneous config across host and virtualization platforms
6. Safe cleanup on command cancellation (ie, very safe/sane command line behavior)
There's much more to add but this is nice short list. [UPDATE] Generally speaking it takes the idea of using VMs locally to guarantee a development environment/state and streamlines the process enormously.
* Most people underestimate the number of ways that you can screw up network config on local VMs.
1. Single command setup across virtualization platforms
2. Tight integration with provisioning tools (ships local config to VM and runs)
3. Network configuration*
4. Automated shared folders (includes NFS since VBox has really bad shared folders)
5. Generally homogeneous config across host and virtualization platforms
6. Safe cleanup on command cancellation (ie, very safe/sane command line behavior)
There's much more to add but this is nice short list. [UPDATE] Generally speaking it takes the idea of using VMs locally to guarantee a development environment/state and streamlines the process enormously.
* Most people underestimate the number of ways that you can screw up network config on local VMs.
The combo of point 2 and 4 is what makes the real difference IMO. Since it's easy to provision a new box with a single command and you don't maintain state on them, you get disposable VMs.
No more going trough a list of VMs and asking yourself if you left any work on that or if you can scrap it. Just `vagrant destroy` and `vagrant up` when you need it again. Plus the provisioning bits gets tested a lot more.
No more going trough a list of VMs and asking yourself if you left any work on that or if you can scrap it. Just `vagrant destroy` and `vagrant up` when you need it again. Plus the provisioning bits gets tested a lot more.
Off topic: please make your site responsive. I really wanted to read your blog from my phone but the current layout makes it impossible.
Anyway, thanks for that article, very informative.
Anyway, thanks for that article, very informative.
I know, it's bad on mobiles, I'll have to change the whole theme :/ I have started to grow tired of the old one, I want to get a cleaner one. I'll try to make some time for that, thanks for reading, I'm glad you enjoyed it!
For anyone coming from veewee, there is a tool for converting templates to packer: https://github.com/mitchellh/veewee-to-packer
And more user-friendly docs for that: http://www.packer.io/docs/templates/veewee-to-packer.html
Can Packer meaningfully work with Docker.io? I gather there is some overlap, but is there a way they can work together? It seems to me:
* Packer abstracts creation of images.
* Vagrant abstracts running images, via a VM, and applying changes to images (via chef/puppet, although that seems to overlap with Packer as well).
* Docker seems to abstract both creation of images (via docker build) and running of images (albeit only on top of LXC).
To complicate things further, it seems like Vagrant has some initial support for running (Linux) images on top of LXC (via Fabio Rehm's plugin) and possibly on top of Docker (although the state of the plugin for this is vague).
All this stuff seems very powerful, but the overlap is confusing. My head hurts...
* Packer abstracts creation of images.
* Vagrant abstracts running images, via a VM, and applying changes to images (via chef/puppet, although that seems to overlap with Packer as well).
* Docker seems to abstract both creation of images (via docker build) and running of images (albeit only on top of LXC).
To complicate things further, it seems like Vagrant has some initial support for running (Linux) images on top of LXC (via Fabio Rehm's plugin) and possibly on top of Docker (although the state of the plugin for this is vague).
All this stuff seems very powerful, but the overlap is confusing. My head hurts...
I'm curious where this fits in with my Vagrant/chef workflow where I load a virtual environment with vagrant and provision both development environments and production environments with chef.
Seems as though there's some overlap between how I provision environments and where this tool fits in. What would be the advantages moving this away from my provisioning tool and towards Packer?
Seems as though there's some overlap between how I provision environments and where this tool fits in. What would be the advantages moving this away from my provisioning tool and towards Packer?
Sounds interesting for sure. I don't see any built-in ability to generate qcow2 style images though. Are you going to add that? There are ways to convert (like from a virtualbox image), but having the built-in ability to create qcow2 would be great. Even better would be having a keystone/glance connector to upload generated images to Openstack!
Yep, definitely.
Packer mentions continuous delivery in the intro documentation[1]. Is there a scenario where you install the result on existing systems or do you rolling deploys with new VMs all the time ? I'm specifically thinking of services that need to maintain state like databases where the second scenario is not very handy.
[1]: http://www.packer.io/intro/use-cases.html
[1]: http://www.packer.io/intro/use-cases.html
[deleted]
Surprised not to see support for KVM.
I could totally use this! The only question I have is can you exclude certain things. I'd want to clone 95% of a production server but then would need to exclude SSL settings and such.
You choose what you want in the final images, so you can exclude whatever you'd like. You basically have complete control over what gets installed, it doesn't hold your hand in any way, really.
This is seriously ambitious! Glad to see it, a tool for cloning existing boxes was the first thing I went looking for when I started using Vagrant.
Not to be confused with https://github.com/keenerd/packer
Is there any PXE boot support with Packer ? How does Packer compare to Cobbler + Koan ?
Is it possible to build AMIs offline? Why is it starting an instance for the source AMI?
It is explained in the opening paragraphs here[1] but I'll explain.
There are two ways to make AMIs: EBS-backed or instance-store. For instance-store, you basically need to debootstrap a chroot or something like that. You can do this offline, technically. EBS-backed you just launch a source AMI, build on that, then re-snapshot it.
For the purposes of a 0.1, Packer currently only supports the EBS-backed approach. This is not a technical limitation, however, and in a future version I do want to add the ability to make instance store.
[1]: http://www.packer.io/docs/builders/amazon-ebs.html
There are two ways to make AMIs: EBS-backed or instance-store. For instance-store, you basically need to debootstrap a chroot or something like that. You can do this offline, technically. EBS-backed you just launch a source AMI, build on that, then re-snapshot it.
For the purposes of a 0.1, Packer currently only supports the EBS-backed approach. This is not a technical limitation, however, and in a future version I do want to add the ability to make instance store.
[1]: http://www.packer.io/docs/builders/amazon-ebs.html
Okay, I guess the question was is it possible to pull down a source AMI without starting an instance, then build the new AMI based on that.
No, that's not planned. I've actually not seen/read anything that does this. If it is possible, it is certainly interesting, but I have a feeling that this would be very tricky.
May I als you why you choose golang, I am just interested im your opinion.
Written in Ruby as well?
Edit: Just Checked GitHub, it is Written in Go, Interesting..... Another sign of Ruby being painfully slow.
Edit: Just Checked GitHub, it is Written in Go, Interesting..... Another sign of Ruby being painfully slow.
I doubt this is why Go was chosen over Ruby, it would be fast enough for this problem. It probably has more to do with distributing the software.
Distributing programs in written in Go is _much_ easier that distributing them in Ruby, i.e. cross compile and ship the binary, but distribution of Ruby programs is hard.
Too much time is wasted on 'omnibus' projects to distribute Ruby programs, just look at Chef or Sensu or Mitchell's own Vagrant installers project:
https://github.com/mitchellh/vagrant-installers
(I don't mean to say those projects were a waste of time, but that it seems like this should be a 'solved' problem).
Distributing programs in written in Go is _much_ easier that distributing them in Ruby, i.e. cross compile and ship the binary, but distribution of Ruby programs is hard.
Too much time is wasted on 'omnibus' projects to distribute Ruby programs, just look at Chef or Sensu or Mitchell's own Vagrant installers project:
https://github.com/mitchellh/vagrant-installers
(I don't mean to say those projects were a waste of time, but that it seems like this should be a 'solved' problem).
Ding ding.
There are other reasons but this was a _big_ one.
There are other reasons but this was a _big_ one.
This seems to me to be an issue that there is some disagreement on between devs and ops people - most devs are happy to distribute software with a Gemfile and see it as a solved problem. Ops packages tend to go the omnibus route like Chef has, or use Jruby and package as a Jar file like http://logstash.net/
I wonder if there is anything developers can do as a community to help solve the problems these distributors feel they have?
I wonder if there is anything developers can do as a community to help solve the problems these distributors feel they have?
neat!
I want to stress that this is the _first public release_. There is a lot of work to be done and a lot of improvements to be made. It may not be "production ready" but it is definitely "experimentation ready." Although I know many companies already starting to use this in production.
I also want to note that although I made Vagrant as well, Packer is not tied to Vagrant in ANY WAY. Packer stands on its own as an extremely useful tool. However, Packer can be used to make Vagrant boxes, as well.
ENJOY!