Shell Scripts vs. Ansible(devopsu.com)
devopsu.com
Shell Scripts vs. Ansible
http://devopsu.com/blog/ansible-vs-shell-scripts/
15 comments
I was on the same boat until Ansible. It's so simple to use. If you have a couple of days to spare, you will pretty much know almost everything you need to. It also doesn't require setting up any infrastructure so you don't have to worry about spinning up some master server.
I totally understand where you're coming from. That's actually why I wrote this. Many of the existing tools just don't have the ROI for smaller systems. The exciting thing is that there are new tools to choose from that have dramatically lower barrier to entry. I'd encourage you to check them out when you have a chance. Also, would love to hear if you ultimately use one, please feel free to email me.
disclaimer: author of project here.
Achieving a middle ground was one of the points of creating the project -- making configuration tools more accessible to everybody, that you can forget and come back to in six months and still know how to work with.
I definitely would be interested in your feedback after your trying things, but I think that's exactly the point Matt was trying to make in the article.
Achieving a middle ground was one of the points of creating the project -- making configuration tools more accessible to everybody, that you can forget and come back to in six months and still know how to work with.
I definitely would be interested in your feedback after your trying things, but I think that's exactly the point Matt was trying to make in the article.
Well the proof of the pudding is in the tasting... I will have to give it a whirl I suppose. Chef/puppet/etc seem easy on the surface when you're getting your feet wet. I find that the complexity monster rears its head when you have to start supporting multiple OSs and combinations of configuration options. Shell scripts fail in this area too but it's generally much easier to find the problem than a morass of metadata files, recipes, and specifications. I would be more than happy to provide feedback (being the curmudgeon that I tend to be).
I appreciate attempts at reducing complexity. I've been meaning to look into Marelle a bit (http://larsyencken.github.io/marelle/) as I find logic programming and CBS to be where CMs should be heading. Is Ansible somewhere along that spectrum?
I appreciate attempts at reducing complexity. I've been meaning to look into Marelle a bit (http://larsyencken.github.io/marelle/) as I find logic programming and CBS to be where CMs should be heading. Is Ansible somewhere along that spectrum?
Ansible doesn't seek to be programming, so my guess is no. I generally am not a fan of that syntax and would disagree that that is the way most administrators and developers brains work -- but I guess I can see it for some who do think that way.
Multiple OSes are pretty straightforward though. Look at the 'group_by' module for dynamically carving up machines based on operating system, and things like the 'when' statement for introducing conditional behavior.
Variables can also be overriden based on groups, so you can define defaults in one group and then have different settings in other groups.
Multiple OSes are pretty straightforward though. Look at the 'group_by' module for dynamically carving up machines based on operating system, and things like the 'when' statement for introducing conditional behavior.
Variables can also be overriden based on groups, so you can define defaults in one group and then have different settings in other groups.
This article will be preaching to the converted for many of you, but I still see so many folks still doing manual installs and shell scripts even for large companies that I felt I needed to try and convert a few more folks over to saner systems.
Fortunately Ansible has dramatically lowered the learning curve and now is a great time to make the switch. So, if you have friends that need an "intervention" from their manual installs, hopefully this will help you make your case to them :)
Of course there are many great config mgmt tools out there. I singled out Ansible because it's so simple to get started with and I've found that the learning curve is usually the biggest hurdle to adoption.
Fortunately Ansible has dramatically lowered the learning curve and now is a great time to make the switch. So, if you have friends that need an "intervention" from their manual installs, hopefully this will help you make your case to them :)
Of course there are many great config mgmt tools out there. I singled out Ansible because it's so simple to get started with and I've found that the learning curve is usually the biggest hurdle to adoption.
I've used a lot of various configuration management systems over the years - CFengine, BCFG2, Puppet and now Ansible.
Two notes-
1) They all work well for the base cases, and make it much easier to handle hundreds of machines - Being able to say "Ensure libcurl is installed on appserver, and make sure that nagios has these plugins we wrote" is amazing.
But where every system starts to break down, is when we have setups that I have a good reason for, but don't fit into the preconceived notion of how to handle things.
For example, if I want to install a package from source, or if I want to manually wget a .deb from an internal server, then dpkg -i it.
What's nice is that most systems allow some flexibility - You can do 90% of things in the pre-written modules, and break out to do manual commands where necessary.
2) One thing I'd really like to see, though, is a system which operated with even LESS intelligence.
Rather than using YAML files, or any sort of "user-friendly" config, let me define variables and use pure Python to iterate and execute using them.
I'd love to have people give me pre-written modules as classes I can call, but having that exposed in a way I can get at it with native loops, if/then, etc, would make things a lot more straight forward.
Two notes-
1) They all work well for the base cases, and make it much easier to handle hundreds of machines - Being able to say "Ensure libcurl is installed on appserver, and make sure that nagios has these plugins we wrote" is amazing.
But where every system starts to break down, is when we have setups that I have a good reason for, but don't fit into the preconceived notion of how to handle things.
For example, if I want to install a package from source, or if I want to manually wget a .deb from an internal server, then dpkg -i it.
What's nice is that most systems allow some flexibility - You can do 90% of things in the pre-written modules, and break out to do manual commands where necessary.
2) One thing I'd really like to see, though, is a system which operated with even LESS intelligence.
Rather than using YAML files, or any sort of "user-friendly" config, let me define variables and use pure Python to iterate and execute using them.
I'd love to have people give me pre-written modules as classes I can call, but having that exposed in a way I can get at it with native loops, if/then, etc, would make things a lot more straight forward.
for server in webservers:
server.ensure_perms('/var/lib/somefile',755)
if server.name in qa_servers:
server.ensure_line('/etc/profile','EXPORT RUBY_ENV=QA')
If you're keen on using pure Python then you might enjoy using fabric. It's got fewer primitives than ansible, but allows you to write your logic easily.
It runs via SSH and requires no particularly special setup:
http://docs.fabfile.org/en/1.8/
If you're a perl lover then my own tool, Slaughter, allows you to install packages, fetch files, and make use of the large CPAN library:
http://www.steve.org.uk/Software/slaughter/guide/
Both of these tools are more basic than Ansible, but also more flexible, each in their own way.
It runs via SSH and requires no particularly special setup:
http://docs.fabfile.org/en/1.8/
If you're a perl lover then my own tool, Slaughter, allows you to install packages, fetch files, and make use of the large CPAN library:
http://www.steve.org.uk/Software/slaughter/guide/
Both of these tools are more basic than Ansible, but also more flexible, each in their own way.
I hadn't thought about Fabric in forever - Thanks for the reminder, I'll check it out again! It looks like it might be the perfect level of abstraction/automation with a minimum of magic.
Although I wrote a competing tool I have to admit I do like fabric.
Many of my own web-applications use it for deployment, and for that purpose it works wonderfully:
* Login via SSH to upload a tarball, generated via git.
* Deploy on remote machine, with a versioned path.
* Reload services.
Just as simple as it needs to be.
Many of my own web-applications use it for deployment, and for that purpose it works wonderfully:
* Login via SSH to upload a tarball, generated via git.
* Deploy on remote machine, with a versioned path.
* Reload services.
Just as simple as it needs to be.
Yesterday I was playing with ansible, testing to migrate my already made across years scripts.
My vote for the one who puts here an elegant solution for something as simple as:
The first parts is easy, you just have to learn a new DSL because ansible ignores your .ssh/config.
The sed is easy to replace to with a lineinfile:
But... how can I iterate over the remote files?
I don't like my solution, is not elegant:
But I see my old script to setup a daemon in a chroot, and I see "my" ansible version... and I don't know what to tell (the ansible version just multiplies the lines, in a much more verbose way, that finally looking at it, you have to think and concentrate much more, than looking at my old and direct .sh)
At the end... this is not new:
But I wish users of my software where so happy first day as I'm with my ansible experience.
My vote for the one who puts here an elegant solution for something as simple as:
ssh user@host sed -i '/FOO/ s/FOO*/BAR/' /etc/bla/bla-*
The first parts is easy, you just have to learn a new DSL because ansible ignores your .ssh/config.
The sed is easy to replace to with a lineinfile:
But... how can I iterate over the remote files?
I don't like my solution, is not elegant:
- name: Get the list of remote files
raw: printf '['; printf "'%s'," /etc/sysconfig/network-scripts/ifcfg-*; printf ']'
register: p
- name: Apply the change to the list of files
lineinfile: dest={{ item }} regexp=^NETWORKING_IPV6 line='NETWORKING_IPV6=no'
with_items: p.stdout
But I see my old script to setup a daemon in a chroot, and I see "my" ansible version... and I don't know what to tell (the ansible version just multiplies the lines, in a much more verbose way, that finally looking at it, you have to think and concentrate much more, than looking at my old and direct .sh)
At the end... this is not new:
ssh WORD < SCRIPT
But I wish users of my software where so happy first day as I'm with my ansible experience.
And other thing I've learn this weekend.
The documentation is great, but there is no "a reference", you have to read 5 or 6 different (long) pages to have an overview of what the DSL is, but the only reference is for the modules, not for the core.
You can search for ansible playbooks in sites like github, and... even if you will find a few which do what they say, you will find many "test" repos, and you will find that the 95% of the available code (no from ansibleworks, but by third parties) implement very poor deployments, without even following basic best practices or any security in mind.
I (as final user) would like a "cpan" or "rubygems" or whatever of ansible playbooks, more extended than the ansibleworks examples. Where people could share, search, vote, download (ansible get?), etc ansible playbooks.
Just suggestions.
The documentation is great, but there is no "a reference", you have to read 5 or 6 different (long) pages to have an overview of what the DSL is, but the only reference is for the modules, not for the core.
You can search for ansible playbooks in sites like github, and... even if you will find a few which do what they say, you will find many "test" repos, and you will find that the 95% of the available code (no from ansibleworks, but by third parties) implement very poor deployments, without even following basic best practices or any security in mind.
I (as final user) would like a "cpan" or "rubygems" or whatever of ansible playbooks, more extended than the ansibleworks examples. Where people could share, search, vote, download (ansible get?), etc ansible playbooks.
Just suggestions.
Thanks for feedback.
I would suggest looking at github.com/ansible/ansible-examples for a fair amount of examples.
I think it's generally true of most systems that a lot of the community implemented content is suited to the needs of someone who wrote it.
As for content sharing, we might be working on something :) Stay tuned!
I would suggest looking at github.com/ansible/ansible-examples for a fair amount of examples.
I think it's generally true of most systems that a lot of the community implemented content is suited to the needs of someone who wrote it.
As for content sharing, we might be working on something :) Stay tuned!
"The first parts is easy, you just have to learn a new DSL because ansible ignores your .ssh/config."
use "-c ssh" for the native OpenSSH connection and it will read your SSH config. As far as jumphosts and other options, paramiko will not. "-c ssh" is the default on platforms that support ControlPersist (because it's faster), so this means CentOS 6 is still using paramiko as a default at least for now.
use "-c ssh" for the native OpenSSH connection and it will read your SSH config. As far as jumphosts and other options, paramiko will not. "-c ssh" is the default on platforms that support ControlPersist (because it's faster), so this means CentOS 6 is still using paramiko as a default at least for now.
wow, direct support, this does not have price, you made my day.
Will research more about the connection options, did go through the available tutorials, and as it did work with the defaults, I did skip the connections part.
Thanks for all the efforts put in this project.
P.S. A way to iterate over remote files could be nice. Something like with_remote_fileglob:
Will research more about the connection options, did go through the available tutorials, and as it did work with the defaults, I did skip the connections part.
Thanks for all the efforts put in this project.
P.S. A way to iterate over remote files could be nice. Something like with_remote_fileglob:
While pure python executions would be nice (and are possible[1]), your examples are easily represented in the yaml files as well:
[EDIT]: Dropped names from tasks to make examples clearer.
---
- servers: webservers
- tasks
- file: mode=0755 path=/var/lib/somefile
...
---
- servers: qa_servers
- tasks
- lineinfile: dest=/etc/profile line='EXPORT RUBY_ENV=QA' state=present
[EDIT]: Dropped names from tasks to make examples clearer.
I know you are nitpicking but the point to using a library or embedded DSL is that you can run completely arbitrary logic inside it. For example, instead of just checking servers to see if they are in the qa_servers list he might have selected the servers that have a name starting with the letter "N" or dynamically fetch a list of server names...
With conditional execution, you can do just that. However that is not really the best way to implement the example posited by the OP.
To nitpick, his example is slightly different. His goes through server by server and then does the 2 lines (if they match the name). Yours does the first task on all the webservers first and then the next part on all the qa_servers.
In theory should produce the same output, but if something relied on doing it the first way that would be the difference.
In theory should produce the same output, but if something relied on doing it the first way that would be the difference.
True, though as you note it's a subtle difference, and in my experience not something I've had to differentiate on.
If you really needed the second task to rely on the output of the prior task, you'd probably want to use conditionals - something you can do without dropping back to the API.
If you really needed the second task to rely on the output of the prior task, you'd probably want to use conditionals - something you can do without dropping back to the API.
If you don't mind using Ruby instead of Python, this sounds a lot like Chef. All my Chef scripts wind up using a bunch of off-the-shelf cookbooks to install Postgres, Nginx, etc., then an app-specific recipe to do the rest. A recipe is just Ruby code (augmented with Chef's "resources"), and you can run code on the server with the ruby_block or bash resource. You can also import cookbooks to define more resources (or LWRPs). If you use chef-solo, librarian-chef, and knife solo, you cut out a lot of the complexity too.
Yep, Chef would be an option if you want "infrastructure as code".
Ansible takes a different track because I want to get out of the practice of writing code. (Different strokes, etc -- but it's intended to be faster to write and review)
Ansible takes a different track because I want to get out of the practice of writing code. (Different strokes, etc -- but it's intended to be faster to write and review)
Chef - Infrastructure as a Code
Ansible - Infrastructure as a Documentation
?> let me define variables and use pure Python to iterate and execute using them.
I think you should give Salt a go. While the usual way to run it is to describe stuff using yaml-via-jinja files, you can also create the same structure using standard python modules. Or if you really don't like the declarative way, just import the existing salt states and fire them the way you described over their api.
I think you should give Salt a go. While the usual way to run it is to describe stuff using yaml-via-jinja files, you can also create the same structure using standard python modules. Or if you really don't like the declarative way, just import the existing salt states and fire them the way you described over their api.
Have a look at Sprinkle. Sprinkle "Installers" are more or less wrappers around shell scripts, so you can write an Installer class that can do just about anything you can do in shell.
https://github.com/sprinkle-tool/sprinkle
https://github.com/sprinkle-tool/sprinkle
Ansible also has a "script" module:
http://www.ansibleworks.com/docs/modules.html#script
http://www.ansibleworks.com/docs/modules.html#script
As someone who has written utilities to check for configuration changes presented by packages systems when they update, glossing over this issue is a bad idea.
Having a unified configuration that you can track and push is good, but if you are installing from RPM or DEB, you want to know if the package is pushing a new config itself, or providing a new config for review.
The utility I wrote looked for .rpmnew and .rpmsave save files and checked with a central repo for whether they were whitelisted (by MD5) or not. Anything not whitelisted resulted in mail being sent alerting admins. Along with an fairly comprehensive per-host auto-update exclude list (which I got yum-autoupdate patched upstream to allow), this allowed us to be fairly confident in allowing our 100+ boxes to auto update every day.
Without paying close attention to packages release notes, interesting things can come down from the maintainers, usually from them back-porting a fix from a later version (or a feature, if it's a RHEL point release).
I think configuration management is a good thing, but I'm not sure it's the best thing to be taught first, unless there's mentors to work with. Some things you just won't think of until it's bit you in the ass, and have that bite be at small scale until you've figured it all out is probably a good thing.
Having a unified configuration that you can track and push is good, but if you are installing from RPM or DEB, you want to know if the package is pushing a new config itself, or providing a new config for review.
The utility I wrote looked for .rpmnew and .rpmsave save files and checked with a central repo for whether they were whitelisted (by MD5) or not. Anything not whitelisted resulted in mail being sent alerting admins. Along with an fairly comprehensive per-host auto-update exclude list (which I got yum-autoupdate patched upstream to allow), this allowed us to be fairly confident in allowing our 100+ boxes to auto update every day.
Without paying close attention to packages release notes, interesting things can come down from the maintainers, usually from them back-porting a fix from a later version (or a feature, if it's a RHEL point release).
I think configuration management is a good thing, but I'm not sure it's the best thing to be taught first, unless there's mentors to work with. Some things you just won't think of until it's bit you in the ass, and have that bite be at small scale until you've figured it all out is probably a good thing.
To address just the package concern, it's worth noting that Ansible can pin packages to a particular version, by naming the version you want installed.
Then, even if a new package is pushed to the repo, you will not receive it (and will be notified by an error by subsequent runs if it's been manually updated) until you update the inventory file.
Many things in Ansible can be pinned in similar ways.
http://www.ansibleworks.com/docs/modules.html#yum
Then, even if a new package is pushed to the repo, you will not receive it (and will be notified by an error by subsequent runs if it's been manually updated) until you update the inventory file.
Many things in Ansible can be pinned in similar ways.
http://www.ansibleworks.com/docs/modules.html#yum
I'd also highly recommend maintaining a local mirror, regardless of your CM system.
Makes things faster and ensures even if you don't pin versions you are not going to get upstream surprises.
Makes things faster and ensures even if you don't pin versions you are not going to get upstream surprises.
A coworker and I needed to get an ElasticSearch server up using chef, and the whole time we were complaining that we wished we could just use shell scripts.
So, we spent a friday and came up with http://fuckingshellscripts.org. At first it was just a parody, but then it started to get real after we found a lot of value in just using shell scripts for simple servers.
Granted, we now use ansible and it has been far easier to implement than chef!
So, we spent a friday and came up with http://fuckingshellscripts.org. At first it was just a parody, but then it started to get real after we found a lot of value in just using shell scripts for simple servers.
Granted, we now use ansible and it has been far easier to implement than chef!
Oh man I thought fuckingshellscripts.org was going to be like giveupandusetables.com (which is now down).
I want to just use fucking shell scripts to configure servers. But it's a gem too which I don't want to install.
I want to just use fucking shell scripts to configure servers. But it's a gem too which I don't want to install.
I think your ansible example could be improved by removing all the names.
- name: Ensure https support for apt is installed
apt: pkg=apt-transport-https state=present
// add one to x
x += 1;
That's a great point. I initially didn't have comments for any of those code sections. However, most of my reviewers encouraged me to add the comments so it would be clearer what each step was doing for beginners.
In that case, you can use YAML comments rather than ansible names. It makes it much more clear that it's optional, and you get the actual code in the log rather than the name/comment.
Ansible actually encourages the use of the names because they show up as task headers.
These are especially valuable when managing a large number of hosts.
These are especially valuable when managing a large number of hosts.
That's exactly why I actively discourage the use of names. I'd much rather see "apt: pkg=foo" than "Ensure foo is installed" in my logs. Especially when I have a large number of hosts.
This is a bit OT because the article is about configuring servers, but I wonder if anyone has found CM systems like this helpful for maintaining their desktop or laptop system configurations?
Every few upgrades of Ubuntu I seem to get into a situation that I want or have to do a clean install, then waste time getting the new system back into a workable state. It would have to be pretty lightweight and intuitive for my purposes.
Every few upgrades of Ubuntu I seem to get into a situation that I want or have to do a clean install, then waste time getting the new system back into a workable state. It would have to be pretty lightweight and intuitive for my purposes.
Great question.
Github has a project called Boxen that does this for Mac OSX: http://boxen.github.com/
Google uses their own Puppet solution for their developer workstations: http://www.youtube.com/watch?v=YlKXRdSAZhY
Ansible will work great for this too - particularly if you're on Linux.
Github has a project called Boxen that does this for Mac OSX: http://boxen.github.com/
Google uses their own Puppet solution for their developer workstations: http://www.youtube.com/watch?v=YlKXRdSAZhY
Ansible will work great for this too - particularly if you're on Linux.
I've done some digging around for this a while back and the only thing that made sense to me was the following (and note that this is my personal setup. It could use work, but this has done fine through Linux reinstalls):
* Keep your dotfiles somewhere that can be synced back easily (e.g. Dropbox, VPS with rsync, git repo, whatever)
* Make all dotfiles symlinks to the above. This way, if anything were to blow your hard drive up, it will have a relatively painless recovery.
* Keep workspace stuff/code/things not too easily replaceable in a separate physical drive if possible. Mount this drive to whatever directory that would make sense to you through your fstab or whatever method you prefer. Adding the nofail option in your fstab is also a good idea if you won't have it connected all the time. Reason for all this: You can now wipe your Linux HDD more confidently if you ever need to do so. Important dotfiles and configs should all be safe, code is on a separate drive and other stuff is backed up off-site hopefully.
* Keep your dotfiles somewhere that can be synced back easily (e.g. Dropbox, VPS with rsync, git repo, whatever)
* Make all dotfiles symlinks to the above. This way, if anything were to blow your hard drive up, it will have a relatively painless recovery.
* Keep workspace stuff/code/things not too easily replaceable in a separate physical drive if possible. Mount this drive to whatever directory that would make sense to you through your fstab or whatever method you prefer. Adding the nofail option in your fstab is also a good idea if you won't have it connected all the time. Reason for all this: You can now wipe your Linux HDD more confidently if you ever need to do so. Important dotfiles and configs should all be safe, code is on a separate drive and other stuff is backed up off-site hopefully.
I use Ansible to manage my laptop: https://github.com/uggedal/playbooks/
I set up my latest work laptop with puppet - reusing my own set of classes from my servers in part.
You have to be wary when apt-get is only one command away but in general it works quite well - I'm not editing so many global configs and got most of them in my dotfiles repo anyway.
You have to be wary when apt-get is only one command away but in general it works quite well - I'm not editing so many global configs and got most of them in my dotfiles repo anyway.
> Which method supports easily templating your configuration files?
But you don't actually use templating. You just copy the file over.
What do you do when you have to edit the configuration file? When it needs to be different on every machine for whatever reason (IPs to listen on, hostnames)? What is the smart way to do that automatically?
But you don't actually use templating. You just copy the file over.
What do you do when you have to edit the configuration file? When it needs to be different on every machine for whatever reason (IPs to listen on, hostnames)? What is the smart way to do that automatically?
Templates and host variables.
Templates use the jinja2 templating engine, so variables are represented by `{{ variable_name }}`, and you can put variables (as well as defaults) into Ansible configs to set them to different values for different servers.
Templates use the jinja2 templating engine, so variables are represented by `{{ variable_name }}`, and you can put variables (as well as defaults) into Ansible configs to set them to different values for different servers.
In my opinion, ansible only beats puppet here in ease of initial setup. I even find the equivalent puppet script for this task to be a tad bit clearer. But we're talking just syntax now. I only see now just how similar puppet and ansible (and perhaps chef too) are. What with the ensure'ing and all.
Puppet is a bit difficult to setup (not too much, mind you, it does its own key generating and being a PKI on first setup just fine, and you can run it from its own internal webserver until you have to go all "webscale"), but after that it hums along nicely. Haven't had too much complaining to do.
Except that they changed the string escaping rules somewhere in the 2.7.x branch, which debian of course pushed because it's in the same point release. That was fun.
Puppet is a bit difficult to setup (not too much, mind you, it does its own key generating and being a PKI on first setup just fine, and you can run it from its own internal webserver until you have to go all "webscale"), but after that it hums along nicely. Haven't had too much complaining to do.
Except that they changed the string escaping rules somewhere in the 2.7.x branch, which debian of course pushed because it's in the same point release. That was fun.
Ansible has two things in its favor (of course, IMHO) when compared to puppet:
It only needs to be installed on the node you're working from, not every node (by default).
Dependency is determined by order in the file, not by a dependency graph.
It only needs to be installed on the node you're working from, not every node (by default).
Dependency is determined by order in the file, not by a dependency graph.
Maybe it's me, but I found Puppet very difficult to get started with. To the point where I would happily use a less capable system that was easier to learn.
That's interesting. I was setting up a server the other day and wanted to document it in a repeatable way but, having used Chef and Anisble in the past made me not want to go through the hassle of setting up a whole "thing"...
So I looked through the Puppet docs for about 5 minutes and found I could just run "Puppet apply whatever.pp" on the server and got everything working in about 15 minutes. And I'd never used it before in my life. So for now, I'm quite impressed with the ease of Puppet.
Chef and Ansible both are really great for 80 to 90% of what I've needed to do in the past. But that last 10-20% is really painful. It requires huge contortions, or poking through the source code so that you understand the details of the system so thoroughly that you can write an new module, or abuse the existing modules and bend them to your will. I suspect that, in the end, Puppet will have the same sorts of limitations as the others. But it did have a very nice way of setting things up that didn't require a whole infrastructure.
So I looked through the Puppet docs for about 5 minutes and found I could just run "Puppet apply whatever.pp" on the server and got everything working in about 15 minutes. And I'd never used it before in my life. So for now, I'm quite impressed with the ease of Puppet.
Chef and Ansible both are really great for 80 to 90% of what I've needed to do in the past. But that last 10-20% is really painful. It requires huge contortions, or poking through the source code so that you understand the details of the system so thoroughly that you can write an new module, or abuse the existing modules and bend them to your will. I suspect that, in the end, Puppet will have the same sorts of limitations as the others. But it did have a very nice way of setting things up that didn't require a whole infrastructure.
Hmm, I haven't heard that feedback before. I'd be very interested in hearing what the last 20% you found complicated was. Drop me an email at michael AT ansibleworks DOT com if you like.
This article doesn't do a very good job of stating it's thesis: The ansible version is no fewer lines than the shell script version; but requires you to understand ansible and how it works.
It doesn't seem preferable to me, from this example.
Note, I don't need to be convinced of the benefits of configuration management in terms of reproducibility and reliability. I'm just saying the OP's examples don't really make the case it suggests they make, of showing how ansible is so much easier than shell scripts. It does not appear to be.
It doesn't seem preferable to me, from this example.
Note, I don't need to be convinced of the benefits of configuration management in terms of reproducibility and reliability. I'm just saying the OP's examples don't really make the case it suggests they make, of showing how ansible is so much easier than shell scripts. It does not appear to be.
I think Matt did a pretty good job of showing how they were at about the same level of complexity for a new audience, but one of the things you don't see is that
(A) there's error handling,
(B) the system will only run commands as needed
and of course using things like 'with_items' and 'notify' ends up making it a lot shorter.
But yeah, most of Hacker News already has a read on that :)
(A) there's error handling,
(B) the system will only run commands as needed
and of course using things like 'with_items' and 'notify' ends up making it a lot shorter.
But yeah, most of Hacker News already has a read on that :)
Does ansible have some sort of REPL so I can use it for both manual administration and automated? I use shellscripts for a lot of stuff because they're just bash one-offs with seniority.
There is an unofficial one: https://github.com/dominis/ansible-shell
The important concept here is Reentrancy [1] and versioned code.
If the configuration is small enough it's easier to write a script in bash that approximates the re-entrancy well enough instead of having to bootstrap the configuration manager, distribute the description files and execute it. In fact a bash library could go a long way covering the scope of these tools.
[1]: https://en.wikipedia.org/wiki/Reentrancy_%28computing%29
If the configuration is small enough it's easier to write a script in bash that approximates the re-entrancy well enough instead of having to bootstrap the configuration manager, distribute the description files and execute it. In fact a bash library could go a long way covering the scope of these tools.
[1]: https://en.wikipedia.org/wiki/Reentrancy_%28computing%29
Can anyone discuss their experience with Ansible versus Chef + knife-solo?
I found Chef clumsy and hard to set up on its own (its central server requirement seems like it was designed just to support a business model), but with knife-solo it's extremely usable.
I found Chef clumsy and hard to set up on its own (its central server requirement seems like it was designed just to support a business model), but with knife-solo it's extremely usable.
I've used both, it's similar but a main difference is that you kick ansible off from another (say your dev machine) which then ssh's in and does it's stuff, whilst knife-solo starts from the host.
> knife-solo starts from the host.
Huh? You run `knife solo cook example.com` on your local machine. It then sshs in and runs what it needs to. (Granted, it runs everything via chef-solo, but the user experience is running a command locally.)
Huh? You run `knife solo cook example.com` on your local machine. It then sshs in and runs what it needs to. (Granted, it runs everything via chef-solo, but the user experience is running a command locally.)
Oh I used it where the host boots up and then it it runs solo on the local machine. So in your example it's similar.
What seems to be missing is _updating_ the nginx.conf with sed or something.
In the article, you're just copying a new magic configuration from somewhere vs. "proper" sed use (in reality, it can be a bit more than that) in the shell script.
In the article, you're just copying a new magic configuration from somewhere vs. "proper" sed use (in reality, it can be a bit more than that) in the shell script.
Initially I was using Ansible's `lineinfile` module to do exactly that. It's pretty trivial to do if you want to compare true Apples to Apples. Here's an article with some examples: http://fabian-affolter.ch/blog/the-lineinfile-module-of-ansi...
However, I wanted to show the actual process I went through. Editing the file in place seemed silly to do with Ansible, so I went with putting the config file in version control and copying it over. That also had the benefit of showing how Ansible encourages better practices than shell scripts.
Of course, that's totally subjective, so for some folks that use shell scripts, they may already know to put their config files in version control. However, in practice, I've not seen that be the case.
However, I wanted to show the actual process I went through. Editing the file in place seemed silly to do with Ansible, so I went with putting the config file in version control and copying it over. That also had the benefit of showing how Ansible encourages better practices than shell scripts.
Of course, that's totally subjective, so for some folks that use shell scripts, they may already know to put their config files in version control. However, in practice, I've not seen that be the case.
Also worth noting, ansible comes with a 'template' module which allows you to generate custom files with variables.
Proper way is to install a known version of nginx and use a known conf, not editing some file which might be different based on linux distro or package maintainer.
Problem is sometime platforms differ a bit, so it's better to patch the config file.
For instance (I know there's vagrant and stuff), I develop on a Mac but deploy on linux. My sed handles that pretty well.
For instance (I know there's vagrant and stuff), I develop on a Mac but deploy on linux. My sed handles that pretty well.
While possible to patch using Ansible (lineinfile), it's typically easier to use templates with host vars instead. i.e. define a set of default values which are for your deploys, and then a host specific variable for your local machine which provides the alternate values.
Does it beat sed? Probably not in the short term, but it makes it easier to not forget to sed a file when deploying (not to mention adding the ability to skip changing the remote file and sighup nginx when the file hasn't changed).
Does it beat sed? Probably not in the short term, but it makes it easier to not forget to sed a file when deploying (not to mention adding the ability to skip changing the remote file and sighup nginx when the file hasn't changed).
The platforms may differ, but a known version should be able to use the same config on the different platforms.
Just looked up a real "sed" example in linux vs linux.
DigitalOcean ships VM that don't have an admin group. So when I tried DO, I added:
grep -q 'admin' /etc/group || groupadd admin
Or, for changing the sshd port:
sed -i -r 's/^#?Port [0-9]*/Port '"$PORT"'/' /etc/ssh/sshd_config
Works great on all platforms... regardless of the default sshd_config.
DigitalOcean ships VM that don't have an admin group. So when I tried DO, I added:
grep -q 'admin' /etc/group || groupadd admin
Or, for changing the sshd port:
sed -i -r 's/^#?Port [0-9]*/Port '"$PORT"'/' /etc/ssh/sshd_config
Works great on all platforms... regardless of the default sshd_config.
You realise that the tools like ansible are made for making sure the groups and users you specify in your config are going to be on the server? That's the point of them. Don't rely on a presetup environment, you'r ansible scripts should setup the environment for everything you need.
If the compile options in both platform matches.
And nginx is full of compile options and modules to enable and disable, so maybe "the same config in the same version" does not work on different platforms.
And nginx is full of compile options and modules to enable and disable, so maybe "the same config in the same version" does not work on different platforms.
Then that's not a same known version is. Nginx doesn't support different loadable modules so compiling it differently make them different versions.
If you're reliant on modules being in there you compile it yourself with a source deb/rpm/etc from nginx. If you don't need a module then don't compile it in.
If you're reliant on modules being in there you compile it yourself with a source deb/rpm/etc from nginx. If you don't need a module then don't compile it in.
Augeas is a safer way of updating config files http://augeas.net/, though I have to admit I've mostly ended up just using sed myself.
Lenses are hard for people to write and error handling is poor though, so it can be very frustrating. The lineinfile method does work fine for simple files, and there's also an INI file module.
In most cases, we recommend using the 'template' module for this reason, in cases where you know what you'd like to put down for the file contents.
In most cases, we recommend using the 'template' module for this reason, in cases where you know what you'd like to put down for the file contents.
The title of this article is really tilting. Does anyone defend shell scripts anymore?
When my shell scripts become untenable I happily hire someone who is an expert at installing packages on a cluster of machines.
It's a complex problem and CMs so far are complex enough that I'm put off by how dense the CM language is. Every time I come across chef my ears start leaking precious brain fluid. Recipes, cookbooks, knives, bags, dependencies, acronyms I haven't bothered googling for... there's enough there that someone can dedicate their time to mastering it and be paid a decent salary.
No thanks, I'm already spending enough time as it is being an expert in something else. I'll just stick with shell scripts because at least I can understand those without much effort. If they become a problem I'll hire someone to fix it. :)
update: That was a little rant-y... after reading this article I might try Ansible if I get the chance. The problem I have with CMs is that they tend to be very complex because they solve a complex problem. Shell scripts tend to be simple (but crude by virtue). I am curious whether there is a middle-ground.