HireFire - The Heroku Worker Manager(github.com)
github.com
HireFire - The Heroku Worker Manager
https://github.com/meskyanichi/hirefire
19 comments
Very handy. Was looking at using http://www.simpleworker.com/ as an alternative to the uncost effective Heroku DJ's but hopefully HireFire will sway me back to using Heroku.
Hey. I was actually also using SimpleWorker prior to HireFire, but I find that setting it up is a bit hard / takes long since you have to set isolated code / credentials / etc to their servers in order to process background jobs. Also, the jobs aren't guaranteed to run immediately after they are queued, may take up to 10 minutes on the cheapest plan. When deployed to Heroku it's easy since Heroku will just use the same compiled slug it uses for Dyno's, no additional code/configuration required, it just works. I also found that on SimpleWorker a job that took only 3 seconds on Heroku, would run for 30-90 seconds on SimpleWorker (no idea why).
That said. There are some advantages when using SimpleWorker such as the web interface, scheduling recurring jobs, and other useful options. Especially when you aren't running on Heroku, and memory is an issue if you have a small VPS. In that case, offloading background jobs to SimpleWorker is ideal. But I find that HireFire will make Heroku Workers cheaper than SimpleWorker due to the fact that jobs appear to take (a lot) longer to finish on SimpleWorker than on Heroku (probably due to the start up time? I'm not sure why).
That said. There are some advantages when using SimpleWorker such as the web interface, scheduling recurring jobs, and other useful options. Especially when you aren't running on Heroku, and memory is an issue if you have a small VPS. In that case, offloading background jobs to SimpleWorker is ideal. But I find that HireFire will make Heroku Workers cheaper than SimpleWorker due to the fact that jobs appear to take (a lot) longer to finish on SimpleWorker than on Heroku (probably due to the start up time? I'm not sure why).
Couldn't agree more. The setup time of SimpleWorker is alot ( especially when dealing with specific gems ) and the fact that for DB work you have to create a separate non concurrent connection to the DB per job makes those times you talk about a reality. Ok, you could use the 'expensive jobs' but Heroku's DJ's win hands down in all tests I've done. Plus who doesn't like the handle_asynchronously :function.
Exactly. :) The DB connection is also quite frustrating depending on the location of the DB. MongoDB with MongoHQ or MongoMachine is fine since it's external already. But otherwise you have to know how to configure for example your MySQL PostgreSQL instances to allow incoming connections from external locations, and also becomes potentially insecure depending on your sysadmin skills.
I find Heroku + DJ + HireFire so much simpler, either using that handle_asynchronous method, or just called delay.my_method(*args) is brilliant. No need to think about isolated code or credentials due to the environment variables that are already set on Heroku.
I should look in to integrating Navvy and Resque as well.
I find Heroku + DJ + HireFire so much simpler, either using that handle_asynchronous method, or just called delay.my_method(*args) is brilliant. No need to think about isolated code or credentials due to the environment variables that are already set on Heroku.
I should look in to integrating Navvy and Resque as well.
Hey Guys, I'm a co-creator/founder of SimpleWorker.com. Can't say enough how appreciative we are of the feedback both good and bad. We've been experiencing pretty strong growth and through this are identifying the sticking points and working through them as fast as we can.
The isolated code issue has been a big issue, and we're addressing this through calls like merge_gem(), and soon merge_mailer(). It's a challenging task, and we see the gaps, but we're improving every day.
Runtime: this is also a top priority. We're working with each and every customer to ensure that their workers execute quickly. 3 seconds vs 30-60 is totally unacceptable and we won't leave you hanging here.
Queue speed: we're prototyping alternatives to http queuing which will dramatically speed up queue times.
This is just the tip of the iceberg for roadmap... Great work on HireFire. I see cases for both. Potentially massive parallelization on SimpleWorker (hundreds if not thousands of concurrent jobs), advanced scheduling needs, etc.
Anyways - we're listening - so any and all feedback is well received. If anybody has needs outside HireFire, in conjunction with, or otherwise, shoot me a line and we'll get you up and running at no cost to start.
Chad Arimura chad[at]simpleworker.com
http://www.simpleworker.com
The isolated code issue has been a big issue, and we're addressing this through calls like merge_gem(), and soon merge_mailer(). It's a challenging task, and we see the gaps, but we're improving every day.
Runtime: this is also a top priority. We're working with each and every customer to ensure that their workers execute quickly. 3 seconds vs 30-60 is totally unacceptable and we won't leave you hanging here.
Queue speed: we're prototyping alternatives to http queuing which will dramatically speed up queue times.
This is just the tip of the iceberg for roadmap... Great work on HireFire. I see cases for both. Potentially massive parallelization on SimpleWorker (hundreds if not thousands of concurrent jobs), advanced scheduling needs, etc.
Anyways - we're listening - so any and all feedback is well received. If anybody has needs outside HireFire, in conjunction with, or otherwise, shoot me a line and we'll get you up and running at no cost to start.
Chad Arimura chad[at]simpleworker.com
http://www.simpleworker.com
Hey Chad, great to hear! I also (despite having created HireFire) still have use-cases for SimpleWorker, however in my case they will likely be for my VPS' rather than on Heroku. On Heroku (even though there isn't a UI, unless you use Resque), it's already a partially managed service to host workers, it runs from the same compiled slug as the dyno's use so all the credentials / env variables are all shared among dyno's and workers, no need to re-apply configuration and such.
I think for me the hardest part was getting it to work the first time, waiting quite a while for the jobs to run to see if I managed to get it to work. The merge_gem is a huge improvement, though, handling credentials and re-connecting databases and other things are quite a hassle to set up (imo). I have to say that once it's all set up, and if the 30-60 start-up issue is gone, that it does seems like a beast for processing a lot of jobs concurrently and is exciting.
I'm definitely still looking forward to seeing future progress of SimpleWorker. So please do keep up the good work!
I think for me the hardest part was getting it to work the first time, waiting quite a while for the jobs to run to see if I managed to get it to work. The merge_gem is a huge improvement, though, handling credentials and re-connecting databases and other things are quite a hassle to set up (imo). I have to say that once it's all set up, and if the 30-60 start-up issue is gone, that it does seems like a beast for processing a lot of jobs concurrently and is exciting.
I'm definitely still looking forward to seeing future progress of SimpleWorker. So please do keep up the good work!
Nice work, glad to see heroku API gems getting some love.
I wrote a similar gem back in January and have had good luck with it: https://github.com/freerobby/middle_management
It's fully configurable via environment variables, so you can make changes without redeploying. It also lets you specify a minimum number of workers (even if there are zero jobs) so that you don't have to wait for workers to spin up if you don't want to.
I wrote a similar gem back in January and have had good luck with it: https://github.com/freerobby/middle_management
It's fully configurable via environment variables, so you can make changes without redeploying. It also lets you specify a minimum number of workers (even if there are zero jobs) so that you don't have to wait for workers to spin up if you don't want to.
Thanks. I like your approach of not having to deploy to change the min/max value of workers. Though it's hard to implement (I think) if you're working with a worker/ratio configuration like HireFire. Currently HireFire also supports the Mongoid ODM. I'm looking in to implementing the Resque worker as well so people can use HireFire with Resque with Redis on Heroku as well.
Thanks for your input!
Thanks for your input!
If I could fork your code and get it working with environment variables, would you be interested in pulling that in? (Not sure if you made a deliberate decision to avoid that).
MiddleManagement works great for my needs, but if these libraries serve the same use case(s), I'd rather converge towards one library than have things spread out.
One other thing you may want to add is a retry loop for the heroku API calls. Every now and then, the heroku gem will throw a RestClient::ServiceUnavailable: 503 Service Unavailable exception. In the worst case, I'm not sure if this is best handled in the gem or at the application layer; however it's definitely worth retrying a couple times if you can avoid the blowup completely. I got some feedback from the Boston.rb group on handling this if you're interested: http://groups.google.com/group/boston-rubygroup/browse_threa...
MiddleManagement works great for my needs, but if these libraries serve the same use case(s), I'd rather converge towards one library than have things spread out.
One other thing you may want to add is a retry loop for the heroku API calls. Every now and then, the heroku gem will throw a RestClient::ServiceUnavailable: 503 Service Unavailable exception. In the worst case, I'm not sure if this is best handled in the gem or at the application layer; however it's definitely worth retrying a couple times if you can avoid the blowup completely. I got some feedback from the Boston.rb group on handling this if you're interested: http://groups.google.com/group/boston-rubygroup/browse_threa...
Thanks for the heads up. Seems like a good idea to catch the 503 errors. Also, for now I'll leave all the configuration defined in pure Ruby. Maybe eventually I'll consider using ENV var's to handle that.
Thanks for your feedback!
Thanks for your feedback!
Looks great. Hoping to see resque support on it soon.
One thing this really shows the need for is a real api on heroku. Having to hardcode your username/pass into env variables is pretty ghetto. Hopefully heroku devs are watching and listening. Oauth plz??
One thing this really shows the need for is a real api on heroku. Having to hardcode your username/pass into env variables is pretty ghetto. Hopefully heroku devs are watching and listening. Oauth plz??
Thanks! Looking at Resque and will probably be implementing it fairly soon.
Ha! And yeah, the ENV variables. :)
Ha! And yeah, the ENV variables. :)
For the low-traffic scenario there's also an older, much simpler solution: https://github.com/mtravers/cheepnis
Interesting and light-weight approach. Though, does it support other ORM's or ODM's like Mongoid? Currently HireFire supports ActiveRecord and Mongoid with Delayed Job. Planning to add Resque with Redis support fairly soon.
I was looking for something like this months ago. Thanks for building, and I'm looking forward to (at least trying to) contribute!
Looks extremely cool. Can't wait to kick the tires.
Does anyone know of gems like this that will work on standard VPS servers? I'm launching something that completely depends on Delayed Job for processing data and would love an interface to manage queues and number of workers spun up at a given time. Also, visually watching the queue (a la resque) would be amazing.
The best I've found so far is: http://trevorturk.com/2011/01/21/delayed_job_admin-a-simple-...
The best I've found so far is: http://trevorturk.com/2011/01/21/delayed_job_admin-a-simple-...
Hey. The gem actually has a local environment option which makes it work on a VPS similar to how it works on Heroku. You could try that and see how it works.
https://github.com/meskyanichi/hirefire/blob/develop/lib/hir...
Also, as far as I know that DJ Admin Interface is the only one for DJ that currently exists. If you want to use Resque, I'll be implementing Resque for HireFire soon.
https://github.com/meskyanichi/hirefire/blob/develop/lib/hir...
Also, as far as I know that DJ Admin Interface is the only one for DJ that currently exists. If you want to use Resque, I'll be implementing Resque for HireFire soon.
[deleted]