Creating your own Heroku on EC2(blog.clearbit.co)
blog.clearbit.co
Creating your own Heroku on EC2
http://blog.clearbit.co/ec2-heroku?hn
8 comments
Is it me or when there is talk about a "personal Heroku" the only thing that seems to be there is a git push deployment?
Heroku is much more than that from the deployment to the logging, to the collaboration. What about DB or all the other external systems a modern web app needs (memcache, redis, etc)?
We build custom apps for many different clients which each have their own server/db. Heroku is costly for that kind of setup, but I haven't found anything else that gives me the same peace of mind and sunny weekends.
Heroku is much more than that from the deployment to the logging, to the collaboration. What about DB or all the other external systems a modern web app needs (memcache, redis, etc)?
We build custom apps for many different clients which each have their own server/db. Heroku is costly for that kind of setup, but I haven't found anything else that gives me the same peace of mind and sunny weekends.
It's absolutely true that Heroku is a lot more than what Dokku, Docker & Vagrant provide, especially when it comes to auto-scaling. Heroku is a good choice for many - but it's also good to have alternatives. For us (at Clearbit), it made sense to build our own infrastructure for cost reasons, especially since we're using EC2's spot instances.
To your points though:
* Dokku uses Heroku's buildpacks and auto-detects your stack when you git push
* Dokku-alt supports collaboration & user-permissions
* Logging is automatically handled (dokku logs app-name)
* Support for Memcache/Redis is baked in (dokku redis:create app-name)
* Postgres is also supported by Dokku, but I personally go with AWS RDS.
To your points though:
* Dokku uses Heroku's buildpacks and auto-detects your stack when you git push
* Dokku-alt supports collaboration & user-permissions
* Logging is automatically handled (dokku logs app-name)
* Support for Memcache/Redis is baked in (dokku redis:create app-name)
* Postgres is also supported by Dokku, but I personally go with AWS RDS.
> For us (at Clearbit), it made sense to build our own infrastructure for cost reasons, especially since we're using EC2's spot instances.
> * Postgres is also supported by Dokku, but I personally go with AWS RDS.
That's funny - I've costed it, and it's cheaper to run equal sized instances of a DB node than to run RDS, plus you get way more control over the tuning and implementation of your database layer.
> * Postgres is also supported by Dokku, but I personally go with AWS RDS.
That's funny - I've costed it, and it's cheaper to run equal sized instances of a DB node than to run RDS, plus you get way more control over the tuning and implementation of your database layer.
> Heroku is costly for that kind of setup, but I haven't found anything else that gives me the same peace of mind and sunny weekends.
If it's costing you enough that you can hire someone to do devops then do just that, hire someone. If not then you're actually saving money vs the cost of the employee.
If it's costing you enough that you can hire someone to do devops then do just that, hire someone. If not then you're actually saving money vs the cost of the employee.
We have been trying to use a similar setup and the pain point was getting DB wired up properly. Dokku has plugins for different databases, but when a server unexpectedly shuts down, it is a pain to restore the wiring and all the data to the original state.
My exact same experience. I do not evaluate a system with the happy path. You can run a blog, but when you run an entreprise app, there is no room for downtime.
I feel the same. I love the ease of Heroku but the premium cost is high. Given the fierce price competition among VPS providers like digitalocean, I've always wondered why there are no cheaper alternatives to heroku. Is their infrastructure just that hard to duplicate?
For what I'm using digitalocean for, it is only $5ish dollars a month more than heroku but infinitely easier to use as it's an actual linux box I have free reign over. I understand I'm not going to make either of them a lot of money but I can't imagine wanting to save money and deal with heroku when a VPS is so cheap now a days.
So how much money do you lose if your server goes down? The grandparent comment is saying that Heroku is more expensive than VPS, not vice versa.
Check out Openshift (https://www.openshift.com), I am currently using the free tier to host static blogs and static sites; quite frankly, it's not bad
Dokku has most of the key features of Heroku for free and even deploys the same buildpacks.
Because building one is non trivial. It deters competitors from attempting it.
That's the sense I get too.
I built my own personal Heroku with a cheap $40 OVH server, and my custom capistrano script.
"cap deploy" is all I need to deploy everything, run database migrations, etc.
I built my own personal Heroku with a cheap $40 OVH server, and my custom capistrano script.
"cap deploy" is all I need to deploy everything, run database migrations, etc.
In playing around with dokku for use on my VPS I just generally got the impression that it wasn't ready to be used in production. I repeatedly ran into snags or bugs, but so many steps had been automagically taken care of by dokku that I oftentimes didn't even know where to begin to look to try to resolve them. If something like dokku is going to do these things automatically it needs to do so much more reliably, otherwise it's worse than nothing at all.
I do believe dokku (or something else like it) will be the go-to in the future, but I think that's a ways off yet.
I do believe dokku (or something else like it) will be the go-to in the future, but I think that's a ways off yet.
I have to agree with you. Dokku is REALLY neat, don't get me wrong, but after working more or less perfectly for a month or so (Only with 2 projects running) I tried to add a new one and everything went to hell. My old projects started having problems and like you I had no clue where to start. I will keep an eye on Dokku along with things like Deis which I also have high hopes for.
Dokku's definitely a young project and has its teething problems. We've been using a fork of it called Dokku-alt which is getting a bit more love these days - I'd give that a shot.
https://github.com/dokku-alt/dokku-alt
https://github.com/dokku-alt/dokku-alt
I was using heroku initially but went with digital ocean to reduce costs. From my experience with using heroku, it's pretty easy to scale without knowing server-side configs, etc. But depending on your app, getting onboarded with digital ocean or the likes of it is not too cumbersome either. For example, digital ocean gives you already baked in setup for django/python. You can simulate heroku by deploying code using git pull given it's on some git repo (not to say that's the only killer feature heroku has). You may have to be wary of some security pitfalls though. I've noticed default password for postgres db is same for all django/python instances in digital ocean. So if you were to allow connection to db outside from localhost without changing default password of db - that's a big, big security risk! I wonder if there may be some websites on digital ocean already impacted by it.
In summary, you may be expected to learn gunicorn/nginx confgis (depending on the app) and have to be cautions of some security pitfalls if you were to go with IaaS. Depending on the scope of your app the work to get around these tasks may or may not be enough to justify using heroku.
Speaking of Docker on AWS - has anyone had trouble with deployments on micro instances? I'm able to deploy once, then subsequent deployments are killed due to a lack of free memory.
This seems like a great setup that I'd love to try at some point, but I think I'll have to fork over more $$$ if I actually want it to work. :/
This seems like a great setup that I'd love to try at some point, but I think I'll have to fork over more $$$ if I actually want it to work. :/
My experience with micro instances convinced me that they are basically a scam to get people to invest time in AWS. It seems they are designed specifically to throttle CPU and other resources in order to encourage people to use the expensive types of servers.
Digital Ocean works great for me. Not a massive ripoff, and actually works. One thing, if you use a 512mb instance for $5/month, probably want to make sure you create a swap file.
Digital Ocean works great for me. Not a massive ripoff, and actually works. One thing, if you use a 512mb instance for $5/month, probably want to make sure you create a swap file.
Are you using the t2 or t1 micro instance? The t2 is much faster and you can see on CloudWatch if you are being throttled.
I blogged last week about setting up Heroku git push style deploys on a cheap VPS. This is really easy to do, but it is not the Heroku experience with scaling, fault recovery, plugins, etc.
Hopefully not too far off topic, but there are other hosting options to also look at:
Microsoft BizSpark: I was accepted a while back, and I am using it to try to get a business idea off the ground. Basically free hosting for a few years to get new business ideas started. I am using a Linux server. Microsoft Azure is pretty cool, and worth a look.
IBM BlueMix: I recently became interested in BlueMix because I am helping a customer use the IBM Watson system in their application. Anyway, if you are already familiar with the Cloud Foundry command line tools, then there is no learning curve for using BlueMix. You can host an app that uses 512MB RAM for free. As you increase RAM and other resources, then it starts to cost.
Hopefully not too far off topic, but there are other hosting options to also look at:
Microsoft BizSpark: I was accepted a while back, and I am using it to try to get a business idea off the ground. Basically free hosting for a few years to get new business ideas started. I am using a Linux server. Microsoft Azure is pretty cool, and worth a look.
IBM BlueMix: I recently became interested in BlueMix because I am helping a customer use the IBM Watson system in their application. Anyway, if you are already familiar with the Cloud Foundry command line tools, then there is no learning curve for using BlueMix. You can host an app that uses 512MB RAM for free. As you increase RAM and other resources, then it starts to cost.
We've had pretty nice results with Cloud66. Technically it's different: Dokku/Heroku use containers, Cloud66 uses full cloud instances. However, most using Heroku think of it as a simplified deployment mechanism, not really worrying about the underlying engineering details.
I tried this on Digital Ocean a while back. Seemed pretty decent.
I can't speak to databases, because RDS/ElasticCache/etc. are right there so I don't think about it as often as I probably should, but for application-level stuff I've found that Mesos with Marathon (or HubSpot's Singularity, which is really cool) addresses this much better, especially with Mesos's Docker executors. You also have the advantage of being able to use this cluster differently should you so choose--Marathon lets you run apps as does Daniel's Dokku setup, but you can use Chronos as a distributed, fault-tolerant cron system and you can also plug in Slack, ElasticSearch, etc. and reuse the same infrastructure as you need it.
It's also pretty easy to get rolling with, so I wouldn't use difficulty as a factor. =) A little more overhead, but a lot more future-proofing. I haven't (yet) open-sourced my "Mesos in a box" Vagrant stuff, because I haven't extricated it yet from my actual line-of-business stuff, but I built upon some (kind of hacky, much improved now) stuff I blogged about a few months ago:
http://github.com/eropple/mesos-experiments
http://edcanhack.com/2014/07/a-virtual-mesos-cluster-with-va...
This stuff is super interesting to me and I'm always looking for neat ways to use scalable infrastructure, so if anyone wants to chat about this stuff offline, email's in my profile.