How I migrated from Heroku to Digital Ocean with Chef and Capistrano(matteodepalo.github.com)

100 pointsby matteodepalo13 anni fa34 comments

matteodepalo.github.com

How I migrated from Heroku to Digital Ocean with Chef and Capistrano

http://matteodepalo.github.com/blog/2013/03/07/how-i-migrated-from-heroku-to-digital-ocean-with-chef-and-capistrano/

14 comments

Every time I see someone doing something cool with Chef, I think of this: http://progopedia.com/language/chef/

And then I think, "Wow. This person must be a mad genius."

jeromewilson13 anni fa

Anyone who's interested in migrating from Heroku go do a lot worse than giving Cloud 66 a try. All the lovely deployment goodness without the lock in.

AsifRahman13 anni fa

humanspecies13 anni fa

I hate the term cloud computing and all the stupid hype that comes with it.

It's called mainframe computing, the exact same clusterfuck your grandfather used in 1970. With bigger hard drives.

thibaut_barrere13 anni fa

Interesting article I think (as someone who already uses Chef).

That said like I commented on your post, create a proper user instead of running as root, really.

matteodepalo13 anni fa

I totally agree and it's next in the list of improvements I want to do. I just wanted to share this asap until it's fresh in my mind.

Edit

Done! :-)

thibaut_barrere13 anni fa

Sure :-) I will definitely forward this to people new to this, I would have loved to have such a tutorial a couple years back!

evaneykelen13 anni fa

I'm planning to move away from Heroku too, partly because of the routing system issues you mention in your article, but also because I want to sharpen my sysadmin fu. Have you considered https://www.cloud66.com/ because it seems like a nice tradeoff between going "bare metal all the way" and Heroku's comfy environment.

matteodepalo13 anni fa

I've tried their service a few weeks ago, but they had a problem related to my particular Gemfile so I abandoned it for the moment. Their product is great and I'm sure they will iron out these small bugs, but for now it's not a valid option for me.

zagi13 anni fa

DigitalOcean recently became fully integrated with Cloud66.com just last week. It's a great way to easily spin up a new server, and echo's our mantra of simple cloud hosting.

AsifRahman13 anni fa

pplante13 anni fa

I love the idea behind this software. I just wish it worked with Django.

neya13 anni fa

Cloud66 is awesome I should say, they take the pain of having to do all the groundwork yourself, which is so freakin awesome ! Are there any similar providers to cloud66, apart from Heroku? (and Engineyard ofcourse)

danso13 anni fa

This was a very helpful and practical tutorial. I don't know why sysops is a source of perpetual confusion for me but it's good to have as many examples as possible. Could you elaborate on the actual hoops that creating a non-root user was creating for you?

matteodepalo13 anni fa

First of all I didn't know whether to provision my machine with root or not. Scraping through sysadmin resources I found people installing Ruby with the "deployer" user and people using root for that task. When I tried installing Ruby with deployer I got many permission errors, but I realize that was caused in large part by my lack of experience with sysadministration.

Also, running processes with runit as non-root requires extra [care][1] and I just wanted to get everything up and improve my scripts AFTER I knew that my setup was viable.

Edit

One more thing: I couldn't install ruby-shadow, which is needed to handle passwords with chef, due to some errors with the version of ruby I had.

[1]: http://smarden.org/runit/faq.html#user

rschmitty13 anni fa

With rbenv you can install ruby directly to your deploy user, no sudo required as it installs to your home directory

https://github.com/sstephenson/rbenv/wiki/Deploying-with-rbe...

matteodepalo13 anni fa

Thanks, I'll try it as soon as I can :-)

thibaut_barrere13 anni fa

You can deploy the infrastructure with root (or a sudoer), and run the app as a regular user (which is quite often called "deploy" in capistrano recipes).

calgaryeng13 anni fa

If I am deploying an application with "deploy" (i.e. non-priviliged), it is ok to also run Unicorn as deploy? Or should it be run as nobody or www-data ?

pw713 anni fa

Did the same thing 2 days ago! And cannot stress enough how rewarding the experience was. Although I never heard of Digital Ocean so I chose linode, but I'm also very happy with them so far.

kawsper13 anni fa

Linode have stackscripts, so you actually could do without Chef with a well structured bash script.

I have asked Digital Ocean about the possibility for Stackscripts, and I don't think they have planned for it, sadly.

It is nice to order a server, choose a stackscript, and have a server to deploy to without even logging in on the instance.

matteodepalo13 anni fa

Author here, I updated the article; now a user called "deployer" runs the app. Turns out it was easier now than when I started with the migration.

genericsteele13 anni fa

This is great. Two weekends ago I did the same type of migration using Linode and Capistrano. Heroku taught us what we should expect from a platform, and it seems like, as a community, we're moving towards getting those great features ourselves.

zagi13 anni fa

This is one of the fundamental differences between a platform and an infrastructure provider. As a platform you build a single tool that is used by everyone and with it come inherent limitations. If an infrastructure provider can give you the building blocks to create a platform service on your own, you maintain full control. So long as your time investment into growing the infrastructure doesn't outweigh the cost savings of a platform you get much more predictable growth, better pricing and maintain full control allowing you to develop a product that's customized to your needs exclusively.

purephase13 anni fa

This is great. Thanks for sharing. I've been toying around with mina, but I may give Capistrano another shot. Right now I have a bunch of build scripts that would be much better suited for Chef and your setup closely mirrors my own.

fomb13 anni fa

So how does this resolve the Heroku routing problem for you? You have one server that internally routes efficiently - but what happens when you scale past one server?

If you're not scaling past one server, why does Heroku's routing cause you issues?

matteodepalo13 anni fa

On Heroku I was using thin that forced me to scale horizontally to handle more than one request at the same time. With this setup I can set a number of unicorn workers bigger than 1 and I'm done. I can scale vertically until I can and then set up more machines with the same methodology and use a load balancer like HAProxy to manage requests.

neilmiddleton13 anni fa

Why not use Unicorn on Heroku?

matteodepalo13 anni fa

There is a limited amount of workers you can run with 512mb of memory (max 3 for my application) and the random routing can overload one dyno while the others are doing nothing. Basically you have to scale and pay more when with Digital Ocean I can buy a 2gb machine and run 10 workers spending 20$ per month.

mark_l_watson13 anni fa

He can use a 2-core VPS with plenty of memory and he will have much more headroom than a single Heroku dyno.

How many people really have to worry about scaling past a single server? To me, the issue is more about uptime so that where PaaS's get extra credit as far as I am concerned. It is great to be able to go on vacation and not much worry about admin hassles.

grey-area13 anni fa

Using unicorn on a vps is the equivalent of many 'servers' or dynos on heroku, and unicorn is taking the place of the heroku routing here, so this does address the problem as you'd have more control over things like timeout and routing strategy, and have other options too for load balancing. If you do need several servers there is software out there to do load balancing, but with caching you can easily scale a single vps to serve a lot of users. For example HN still runs on one server I think, there's a pic of it somewhere that pg posted...

rmoriz13 anni fa

With https://github.com/rmoriz/knife-digital_ocean you can launch and bootstrap an instance using knife in a single command.

Works with knife-solo and traditional chef-server environments.

JacobJans13 anni fa

I would be very leery of hosting any serious business on Digital Ocean. Their terms of service are simply too strict.

For example, you can't send email to your users without double opt-in.

Yikes!

Not to mention the ambiguous language banning 'offensive' and 'inappropriate' content.

mitchwainer13 anni fa

Hi - It's Mitch from DigitalOcean. We are still ironing out our TOS. Given that we are still a startup we are always constantly looking to improve the language.

We don't consider ourselves too strict. As long as you are not trying to run an attack or spam/flood our network, we are totally cool. :)

JacobJans13 anni fa

Mitch,

I appreciate the sentiment, but I can't stake my livelihood on promises made on an internet forum.

I want to be a Digital Ocean user, but I'm going to base that decision on the legal agreements first.

I do hope this will encourage you guys to remove some of the ambiguity in your TOS, and to remove that double opt-in requirement.

Best,

Jacob

mitchwainer13 anni fa

I totally hear you Jacob. Know exactly where you're coming from.

And yes, we will continue to work on our TOS. :)

Best of luck! Mitch