We started off writing the bulk of our server setup/deployment automation in chef, and have since completely abandoned it
The core problems we had with chef were:
• worse than ruby: the chef ruby DSL is like some bastardized, crippled ruby - e.g. ruby_block{}, just uggh
• way too slow & resource intensive: chef itself uses a lot of memory and CPU, has a slow boot time, and does stuff like execute apt for each desired package on each node on each run. this might work fine if you're running on beefy physical/virtual hardware, but not for managing hundreds/thousands of tiny LXC containers that need to be scaled on-demand in seconds
• not self-hosted: chef seemed to have real difficulty closing the loop and being the thing that deployed and configured itself. there's guides online for scripting yourself up to a basic chef setup, but what if you want your chef client to bootstrap with some custom rubygems? back to bash scripting - and then how does that script get on each of your nodes? chef isn't intended to build/deploy itself the way it does the rest of your stuff
We've now transitioned everything to heroku buildpacks + a build server, which create self-contained "slugs" and therefore are self-hosting (i.e. the build server can build itself), and allows us to have a single build/deploy process for everything
While I'm not a Chef fanboy, far from it, some of these comments are just not true.
"and does stuff like execute apt for each desired package on each node on each run". No you just need to set it up correctly so that it keeps the timestamp of the latest apt-get update, and does not refresh it within the next x hours.
I never got on well with either. Puppet was mind-bogglingly slow (even locally without a master), both Chef and Puppet felt overwrought, and I prefer keeping dependencies to a minimum.
Just a pointer for anyone else that doesn't have much love for either: I found myself happy and productive with Salt. Fast, simple, and lets me do what I needed with less Byzantine setup: http://docs.saltstack.org/en/latest/
There's also Salty Vagrant and Salt Cloud for local and remote provisioning.
The job of an OS not a human admin. "apt-get install puppet" on the clients and "apt-get install puppetmaster" on the puppetmaster. That's about it.
"Puppet was mind-bogglingly slow"
Was it a pause exactly equal to one DNS lookup timeout? The SSL inside puppet used to get all wound up about reverse DNS matching the forward or whatever exactly. You need working DNS to puppet. If DNS is dead you may as well forget debugging puppet until your local DNS is healthy.
Also its possible to do unusual SSL configurations that can make it a bit slow. Vanilla out of the box should be reasonably fast. Starving a virtual image of CPU can make the SSL slow... a virtual 40 MHZ 386 equivalent is not going to do SSL any faster than a physical 40 MHZ 386 used to.
Came here to comments to post about saltstack. Couldn't agree more. So blindingly fast. I think people who comment that puppet isn't slow just don't know how fast salt is. Tens of thousands of clients polled in just seconds is something puppet just can't approach. I also found not having to learn yet _another_ DSL refreshing.
The short version is: Puppet best fits the way I think about how such systems should work. Despite ostensibly belonging to the same genre of system, the three of them have subtle but very important differences.
There are lots of things I'd like to improve in Leo that involve chucking its current code base (e.g.: back end storage is XML and thus version control hostile; front end is not a web browser; acyclic should get dropped from "rooted, ordered, acyclic graph"), but until that bit of brilliance dawns over the world, I'll continue to use Leo.
Been puppet'ing for years across maybe a hundred machines. Looks like almost everything Phil initially wrote about puppet got edited afterwards.
There is a third and fourth solution to the "Encrypted data bags for puppet" problem. The third, my solution, is to never, ever, store AAA in configuration system. EVER! I do store calls to programs and such or even just data files as a program. I admit sometimes the "program" to get certain passwords is something like "backtick cat somefile backtick" but usually I do better. Those AAA programs/repos are handled much more delicately and securely than a "everything goes" config system that everyone can mess with.
The fourth solution is the implied idea that you'd never rotate AAA credentials on a regular basis and never change infrastructure passwords when someone quits, which sounds pretty funny to me. Hey HN, my mysql root password for a month back in 1998 was: (insert something like line noise here)
Another old time puppeteer observation is everyone has a SSL nightmare eventually and even the mighty GOOG can't help you sometimes. Especially on restoral of backups, or replication of live systems, it can get pretty hairy. Also DNS malfunctions can horribly confuse puppet's SSL occasionally. This is something you'll only hear from an old puppeteer not a short experiment like the article.
Last week I interviewed the product manager of Puppet [0] and asked him to differentiate Puppet from Chef and cfengine. He didn't fall for it, and just said something like "the most important part is that you do something."
I've used both (and cfengine) to varying degrees and would have to agree. Simply using a configuration management tool takes an incredible amount of work off your shoulders, you can't go wrong with either.
I have the feeling that choosing a configuration management system is very similar to choosing a build system. There are a few big players where it's easy to do easy things and possible to do complex things, but hacking with them is very arcane (like GNU make). There seem to be quite a lot of documentation and contributed packages for both Puppet and chef, for example.
But, instead of trying to fix what is broken, it seems that a lot of people end up reinventing the wheel and writing their own "lightweight" configuration management system.
As someone that wants to switch to such a system, I believe that it will be easier to learn puppet since the existing code base is just enormous.
Why not add: or Fabric or Ansible or something else? I haven't found much that Fabric won't do for me and I haven't found much reason to install Ruby on my system [Python is a system package].
For a non-Puppet and non-Chef user, would someone explain why you'd limit your choices to puppet and chef?
One huge advantage that puppet and chef offer is abstraction from the underlying system. So if you write a configuration that says "Apache should be running, have PHP installed, and have the following 3 virtual hosts installed", then you don't care if you're running on Solaris, Debian, or CentOS. The manifest/recipe takes care of figuring out what the package is called and where the config files go.
I'm not familiar with Ansible, but Fabric seems more like SSH with a for loop. Puppet and Chef are not about automating the typing as much as they are about ensuring consistency. In my Apache example above, if someone goes and changes a vhost file by hand, that file will be replaced with a good one and Apache will be restarted, even if that file was made by a complicated template.
I use Fabric to run Puppet. Setting up a puppet server is overkill for my small setup, but I found Puppet to be better suited to managing server configuration and more scalable should my number of servers continue to grow.
I have started using chef to automate build environments on top of vagrant. One thing that made me choose chef is windows support: chef uses winrm on windows, + things like powershell, and that seems to work better than puppet (e.g. I could not for the life of god make a unattended install of visual studio with puppet).
On the - sides, I don't find it particularly well documented (it feels like reading the MSDN, where things are self-referentials), and it is quite slow.
For those looking for a dead simple alternative to Chef or Puppet, I wrote Shoestrap. It's a pure Bash, no-BS set of scripts. Doesn't have all the bells and whistles of Chef, but it might be enough for your needs.
I too wrote my own system - which uses Perl. Mine is called Slaughter and the upcoming 2.x release allows you to fetch policies/instructions from remote servers via one of:
* rsync
* http
* git clone
* hg clone
etc. It is pretty extensible and covers my needs. Beyond that it is a little hard to know.
I've tried both Puppet and Chef. Puppet fits my mental model more, but I had for more success with actually getting Chef up and running.
Puppet, on the surface, has less 'messing about' to do in order to get things running - but my experience was that I encountered a few issues which basically caused me to spend way too much time mucking about rather than working.
Ideally I would like to use Puppet, I feel like it's cleaner and it seems more 'logical' to me at least - but I only have actual experience of Chef working, so make of that what you will!
Once Chef is up and running, it absolutely flies, which is always good!
In our experience puppet is easier to setup and start, chef server stack is HUGE compared to just installing a package for puppet master and certs/knife setup is a PITA.
The DSL is pretty similar except for how they track dependencies and general workflow, where chef is `procedural`, puppet internally builds a call graph based on a declarative syntax which can be hard to track down and understand in some situations.
Regarding cookbooks vs manifests, both have tons of modules around, but like with any plugins/modules software the basic stuff in general is cover, but you will need to get your's hand dirty to get things to work your way and not all manifest/cookbooks are good to use at all..
We don't found compilation and install time to be an issue for us, a complete lamp stack install fully customized takes less than 2 minutes per node.
We have a linux background and didn't find ruby hard at all to hack and mess around to solve some of our issues like custom facters and install some rare stuff like nsis :)
I have used both extensively. I found Puppet much easier to get started with but the custom Puppet DSL becomes very quickly constraining. I know that a new pure Ruby DSL is in the works. I found that Chef has a steeper learning curve but many times greater productivity. I do feel that their is much better tooling support for Chef than Puppet, in large part because Chef's pure Ruby approach is much friendlier to Ruby developers. It remains to be seen whether Puppet's coming Ruby DSL will have the same level of functionality.
If you are managing a small application, Chef may feel heavyweight. However, imho, if you can't manage Service-Oriented Architecture w/out a serious configuration management system, be it Chef or Puppet. There are just too many moving parts.
I pretty much hate puppet after 4 years of dealing with it. Scaling problems, having to run another bunch of crap to store configs, in an app that was designed to store configs.
I'm getting all riled up just remembering the hell.
I migrated all my machines to salt 6 months ago after using Puppet for a few years. It's been pretty painless but I would check out Ansible also before commmiting to a Salt transition.
Over the years, I've also used cfengine and a host of other systems and have decided that simplicity and the ability to customize a system to fit your needs are key. Turns out it's quite easy to roll your own, given the right tools.
I now use:
The reason I chose Puppet was that it allows my servers to pull their configuration instead of my pushing it to them. This is important because my servers are all remote and behind firewalls that I don't control. Using Puppet, my servers can make an outbound connection to the Puppet Master without having to mess with firewall ingress rules.
Things like Salt or Fabric require direct access to the target servers and won't work in my situation. I'm not sure about how Chef works as it wasn't available (popular) when I was setting this up several years ago.
I like the topic, but have to say the article wasn't that great - the author didn't seem to have experience with puppet and had to be corrected in the comments section heavily.
[+] [-] themgt|13 years ago|reply
The core problems we had with chef were:
• worse than ruby: the chef ruby DSL is like some bastardized, crippled ruby - e.g. ruby_block{}, just uggh
• way too slow & resource intensive: chef itself uses a lot of memory and CPU, has a slow boot time, and does stuff like execute apt for each desired package on each node on each run. this might work fine if you're running on beefy physical/virtual hardware, but not for managing hundreds/thousands of tiny LXC containers that need to be scaled on-demand in seconds
• not self-hosted: chef seemed to have real difficulty closing the loop and being the thing that deployed and configured itself. there's guides online for scripting yourself up to a basic chef setup, but what if you want your chef client to bootstrap with some custom rubygems? back to bash scripting - and then how does that script get on each of your nodes? chef isn't intended to build/deploy itself the way it does the rest of your stuff
We've now transitioned everything to heroku buildpacks + a build server, which create self-contained "slugs" and therefore are self-hosting (i.e. the build server can build itself), and allows us to have a single build/deploy process for everything
https://devcenter.heroku.com/articles/buildpacks
[+] [-] dikbrouwer|13 years ago|reply
"and does stuff like execute apt for each desired package on each node on each run". No you just need to set it up correctly so that it keeps the timestamp of the latest apt-get update, and does not refresh it within the next x hours.
"not self-hosted". Somewhat agreed, although you can easily get around this with the excellent https://github.com/tobami/littlechef
[+] [-] Tombar|13 years ago|reply
[+] [-] stock_toaster|13 years ago|reply
[1]: http://wiki.opscode.com/display/chef/Knife+Bootstrap
[+] [-] amalag|13 years ago|reply
[+] [-] jtimberman|13 years ago|reply
You can always drop to regular, plain Ruby, anywhere you like.
[+] [-] pyritschard|13 years ago|reply
[+] [-] Hontano|13 years ago|reply
Just a pointer for anyone else that doesn't have much love for either: I found myself happy and productive with Salt. Fast, simple, and lets me do what I needed with less Byzantine setup: http://docs.saltstack.org/en/latest/
There's also Salty Vagrant and Salt Cloud for local and remote provisioning.
[+] [-] VLM|13 years ago|reply
The job of an OS not a human admin. "apt-get install puppet" on the clients and "apt-get install puppetmaster" on the puppetmaster. That's about it.
"Puppet was mind-bogglingly slow"
Was it a pause exactly equal to one DNS lookup timeout? The SSL inside puppet used to get all wound up about reverse DNS matching the forward or whatever exactly. You need working DNS to puppet. If DNS is dead you may as well forget debugging puppet until your local DNS is healthy.
Also its possible to do unusual SSL configurations that can make it a bit slow. Vanilla out of the box should be reasonably fast. Starving a virtual image of CPU can make the SSL slow... a virtual 40 MHZ 386 equivalent is not going to do SSL any faster than a physical 40 MHZ 386 used to.
[+] [-] nodata|13 years ago|reply
Compilation is way quicker in 3.0, averages a couple of seconds per node for me. Client side is parallel, not sure how it could be quicker.
[+] [-] retrogradeorbit|13 years ago|reply
[+] [-] jacques_chester|13 years ago|reply
I chose Puppet.
http://chester.id.au/2012/06/27/a-not-sobrief-aside-on-reign... (scroll down if you don't enjoy my windbaggery).
The short version is: Puppet best fits the way I think about how such systems should work. Despite ostensibly belonging to the same genre of system, the three of them have subtle but very important differences.
[+] [-] atsaloli|13 years ago|reply
[+] [-] thyrsus|13 years ago|reply
There are lots of things I'd like to improve in Leo that involve chucking its current code base (e.g.: back end storage is XML and thus version control hostile; front end is not a web browser; acyclic should get dropped from "rooted, ordered, acyclic graph"), but until that bit of brilliance dawns over the world, I'll continue to use Leo.
[+] [-] VLM|13 years ago|reply
There is a third and fourth solution to the "Encrypted data bags for puppet" problem. The third, my solution, is to never, ever, store AAA in configuration system. EVER! I do store calls to programs and such or even just data files as a program. I admit sometimes the "program" to get certain passwords is something like "backtick cat somefile backtick" but usually I do better. Those AAA programs/repos are handled much more delicately and securely than a "everything goes" config system that everyone can mess with.
The fourth solution is the implied idea that you'd never rotate AAA credentials on a regular basis and never change infrastructure passwords when someone quits, which sounds pretty funny to me. Hey HN, my mysql root password for a month back in 1998 was: (insert something like line noise here)
Another old time puppeteer observation is everyone has a SSL nightmare eventually and even the mighty GOOG can't help you sometimes. Especially on restoral of backups, or replication of live systems, it can get pretty hairy. Also DNS malfunctions can horribly confuse puppet's SSL occasionally. This is something you'll only hear from an old puppeteer not a short experiment like the article.
[+] [-] smoyer|13 years ago|reply
[+] [-] mpdehaan2|13 years ago|reply
FWIW, front page is http://ansible.cc and there's a FAQ there that explains a bit more.
Docs are http://ansible.cc/docs
[+] [-] ivanvanderbyl|13 years ago|reply
[+] [-] reinhardt|13 years ago|reply
[+] [-] swalberg|13 years ago|reply
I've used both (and cfengine) to varying degrees and would have to agree. Simply using a configuration management tool takes an incredible amount of work off your shoulders, you can't go wrong with either.
0 - http://linuxadminshow.com/2012/10/28/episode-4-puppet/
(edit - I said I interviewed the Chef guy, I meant Puppet)
[+] [-] emillon|13 years ago|reply
But, instead of trying to fix what is broken, it seems that a lot of people end up reinventing the wheel and writing their own "lightweight" configuration management system.
As someone that wants to switch to such a system, I believe that it will be easier to learn puppet since the existing code base is just enormous.
[+] [-] CoffeeDregs|13 years ago|reply
For a non-Puppet and non-Chef user, would someone explain why you'd limit your choices to puppet and chef?
[+] [-] swalberg|13 years ago|reply
I'm not familiar with Ansible, but Fabric seems more like SSH with a for loop. Puppet and Chef are not about automating the typing as much as they are about ensuring consistency. In my Apache example above, if someone goes and changes a vhost file by hand, that file will be replaced with a good one and Apache will be restarted, even if that file was made by a complicated template.
[+] [-] benmccann|13 years ago|reply
[+] [-] account_taken|13 years ago|reply
[+] [-] cdavid|13 years ago|reply
On the - sides, I don't find it particularly well documented (it feels like reading the MSDN, where things are self-referentials), and it is quite slow.
[+] [-] cmer|13 years ago|reply
https://github.com/cmer/shoestrap
https://github.com/cmer/shoestrap-example
[+] [-] stevekemp|13 years ago|reply
* rsync * http * git clone * hg clone
etc. It is pretty extensible and covers my needs. Beyond that it is a little hard to know.
http://steve.org.uk/Software/slaughter/new.html for the upcoming 2.x release.
[+] [-] powatom|13 years ago|reply
Puppet, on the surface, has less 'messing about' to do in order to get things running - but my experience was that I encountered a few issues which basically caused me to spend way too much time mucking about rather than working.
Ideally I would like to use Puppet, I feel like it's cleaner and it seems more 'logical' to me at least - but I only have actual experience of Chef working, so make of that what you will!
Once Chef is up and running, it absolutely flies, which is always good!
[+] [-] m0shen|13 years ago|reply
Some highlights:
• Python based
• Uses SSH key-based auth
• No central server required
• Only requires SSH and a Posix shell on the client
[+] [-] Tombar|13 years ago|reply
In our experience puppet is easier to setup and start, chef server stack is HUGE compared to just installing a package for puppet master and certs/knife setup is a PITA.
The DSL is pretty similar except for how they track dependencies and general workflow, where chef is `procedural`, puppet internally builds a call graph based on a declarative syntax which can be hard to track down and understand in some situations.
Regarding cookbooks vs manifests, both have tons of modules around, but like with any plugins/modules software the basic stuff in general is cover, but you will need to get your's hand dirty to get things to work your way and not all manifest/cookbooks are good to use at all..
We don't found compilation and install time to be an issue for us, a complete lamp stack install fully customized takes less than 2 minutes per node.
We have a linux background and didn't find ruby hard at all to hack and mess around to solve some of our issues like custom facters and install some rare stuff like nsis :)
[+] [-] bryanwb|13 years ago|reply
If you are managing a small application, Chef may feel heavyweight. However, imho, if you can't manage Service-Oriented Architecture w/out a serious configuration management system, be it Chef or Puppet. There are just too many moving parts.
[+] [-] sciurus|13 years ago|reply
http://puppetlabs.com/blog/ruby-dsl/ http://puppetlabs.com/blog/gsoc-project-ruby-dsl-for-puppet/
[+] [-] mgarfias|13 years ago|reply
I'm getting all riled up just remembering the hell.
Next time I get to pick, I'll use salt.
[+] [-] uggedal|13 years ago|reply
[+] [-] jtopper|13 years ago|reply
[+] [-] gyepi|13 years ago|reply
gsl https://github.com/imatix/gsl and synctool https://github.com/walterdejong/synctool
The former is used to generate configurations for various classes of systems and the latter pushes them out over ssh. Works very well.
[+] [-] bwb|13 years ago|reply
Know of another big hosting company using Puppet and they love it, for about 10k+ servers.
[+] [-] Corrado|13 years ago|reply
Things like Salt or Fabric require direct access to the target servers and won't work in my situation. I'm not sure about how Chef works as it wasn't available (popular) when I was setting this up several years ago.
[+] [-] uggedal|13 years ago|reply
[+] [-] krosaen|13 years ago|reply
While on the topic of devops, worth checking out:
http://palletops.com
http://linkedin.github.com/glu/docs/latest/html/index.html