Rails 4 Engines(tech.taskrabbit.com)
tech.taskrabbit.com
Rails 4 Engines
http://tech.taskrabbit.com/blog/2014/02/11/rails-4-engines/
13 comments
I am learning Rails here but I know Django very well. Anybody knowing could give a explanation about what are the differences between Rails engine and Django apps?
It looks like midas007’s comment has an answer: https://news.ycombinator.com/item?id=7225851
Do you do API versioning at taskrabbit?
Take a look at VersionCake ( https://github.com/bwillis/versioncake ). Yes, I am one of the authors.
After having gone down the path of separate API controllers, we learned that the APIs had become a second class citizen and had different code paths than the web based resources.
We unified everything; the same endpoint that the user goes through for web signup is the same endpoint that the API will request for a create.
We're just using respond_to's to respond to the various types of clients that make requests.
Utilizing Rails, Jbuilder, and VersioCake, we have a unified, versioned API that can support earlier client versions (thanks to VersionCake).
After having gone down the path of separate API controllers, we learned that the APIs had become a second class citizen and had different code paths than the web based resources.
We unified everything; the same endpoint that the user goes through for web signup is the same endpoint that the API will request for a create.
We're just using respond_to's to respond to the various types of clients that make requests.
Utilizing Rails, Jbuilder, and VersioCake, we have a unified, versioned API that can support earlier client versions (thanks to VersionCake).
That's a really great use of jbuilder. I thought their only benefit was separating view logic but this really makes them useful. Now if they only had a name that was different from a Java thing.
Yes, there's some fancy business for point release semantic versioning, but something I left out was that the backend API engines all have EVEN MORE directories: apps/account/controllers/account/v3/something_controller.rb
That kind of thing.
That kind of thing.
This looks like a very nice concept, I've been exploring something similar but didn't get quite as far into the engines. What I've been doing is have 1 rails app that does the stuff rails is good at and have smaller Sinatra apps mount into the config.ru for the parts like APIs and the parts that handles web hooks from external services. It's not as clean as your solution though as there is still model sharing.
It's very tempting to do model sharing because it just makes things so easy but I can definitely see having to rewrite models when the time comes to scale out the parts of the app that is getting more hits. With your approach I wouldn't have to worry about that because it's modular from the start.
Would love to give this approach a try! Thx for the article.
It's very tempting to do model sharing because it just makes things so easy but I can definitely see having to rewrite models when the time comes to scale out the parts of the app that is getting more hits. With your approach I wouldn't have to worry about that because it's modular from the start.
Would love to give this approach a try! Thx for the article.
We had a very similar path at TaskRabbit. As we made all those other Rails and/or sinatra and/or node apps, we ran into the issues noted in the "Versus Many Apps" section. Basically: testing, coordination, and deployment were an order of magnitude more complicated. We obviously are now at this engine situation.
My dream: A mechanism that will allow me to avoid having to do a whole-site upgrade of an ancient Rails 2.3.8 codebase on Ruby 1.8.7 by slowly refactoring services out into Rails 4 engines on Ruby 2.1, keeping the entire production site up at all times.
Put a reverse proxy server such as Nginx in front of your existing Rails 2.3 servers as well as a new Rails 4 app server. Slowly, route by route, port controllers over to the Rails 4 application and map traffic there using Nginx rules. This method is completely safe: If you notice discrepancies, you can point traffic back to your Rails 2.3 environment while you improve the behaviour of the Rails 4 replacement. Lather, rinse, repeat until it's all ported over.
Be careful about sessions/cookies if you do this.
I'm not sure that rails 4 sessions can be shared with rails 2 sessions.
I'm not sure that rails 4 sessions can be shared with rails 2 sessions.
I agree this is a good approach, but many companies use something like Heroku, which doesn't allow this kind of fine-grained control.
Kind of scary how few people know how to set up nginx anymore.
Kind of scary how few people know how to set up nginx anymore.
Heroku definitely does. Even inside a single dyno.
Switch to at least 2x-sized dynos and use nginx-buildpack (https://github.com/ryandotsmith/nginx-buildpack) with multi-buildpack (https://github.com/ddollar/heroku-buildpack-multi).
You'll probably need to modify/fork the ruby buildpack to do asset compiles twice (if necessary) and check-in both apps into the same repo. And of course modify the nginx configuration.
Switch to at least 2x-sized dynos and use nginx-buildpack (https://github.com/ryandotsmith/nginx-buildpack) with multi-buildpack (https://github.com/ddollar/heroku-buildpack-multi).
You'll probably need to modify/fork the ruby buildpack to do asset compiles twice (if necessary) and check-in both apps into the same repo. And of course modify the nginx configuration.
You could setup nginx on a micro ec2 instance and have it proxy to heroku.
You should also be able to setup a heroku nginx app that proxies to other heroku apps.
You should also be able to setup a heroku nginx app that proxies to other heroku apps.
shameless self promotion
Which is why I started doing these
http://www.youtube.com/playlist?list=PLjQo0sojbbxUav7I746f0l... /shameless self promotion
http://www.youtube.com/playlist?list=PLjQo0sojbbxUav7I746f0l... /shameless self promotion
I watched your Installing Ruby video to see how you did it, I strongly disagree with using rbenv in production. It adds on a completely unnecessary layer of complexity to the system and adds maintenance steps that are likely to be forgotten by the next fellow.
I recommend using Ubuntu server and using the BrightBox PPA to install 2.1. Updates are an apt-get away, and adding PPAs is far easier to do with configuration management than managing rbenv or rvm.
I use rbenv in development and it's a godsend. I keep it the hell away from production, though.
I recommend using Ubuntu server and using the BrightBox PPA to install 2.1. Updates are an apt-get away, and adding PPAs is far easier to do with configuration management than managing rbenv or rvm.
I use rbenv in development and it's a godsend. I keep it the hell away from production, though.
I've got to confirm the troubles of rbenv in production. I highly encourage alternative solutions and I'm furiously examining BrightBox.
Are there any equivalents to the BrightBox for redhat based systems? I have looked around but found nothing at the moment
I looked too before I made the comment but I couldn't find anything. That's why I recommended Ubuntu.
Getting OS support for brand new software packages is tough. It's not just the package itself that needs support but all the things that depend on it.
If you have to use red hat, then you should probably either stick with 2.0 or just use rbenv until a proper package is released.
Getting OS support for brand new software packages is tough. It's not just the package itself that needs support but all the things that depend on it.
If you have to use red hat, then you should probably either stick with 2.0 or just use rbenv until a proper package is released.
Don't you need rbenv (or similar) in GP's case of migrating old app to new?
You mean my case? Yeah, that and other complexities of the approach will probably keep me from using it. We're talking about a huge app with its own custom extension system with a ginormous amount of technical debt. Just keeping deployment sane would be a mess.
How does the database come into play there? Do you just have both the rails 2 and rails 4 app point to the db at the same time?
Rails app(project):Django project, Rails Engine:Django app
Bonus: directory layouts, since they're conceptually connected:
http://guides.rubyonrails.org/getting_started.html#creating-...
http://www.deploydjango.com/django_project_structure/index.h...
----
The other important point on large scale Rails architecture is that common functionality can/should be extracted to Rails plugins, which can then be turned into Ruby gems.
http://guides.rubyonrails.org/plugins.html
Great Rails project: https://github.com/diaspora/diaspora which uses MVCP (Presenter)
The other bit it to make use of concerns in Rails 4 for extra DRY controllers and models:
https://stackoverflow.com/questions/14541823/how-to-use-conc...
Bonus: directory layouts, since they're conceptually connected:
http://guides.rubyonrails.org/getting_started.html#creating-...
http://www.deploydjango.com/django_project_structure/index.h...
----
The other important point on large scale Rails architecture is that common functionality can/should be extracted to Rails plugins, which can then be turned into Ruby gems.
http://guides.rubyonrails.org/plugins.html
Great Rails project: https://github.com/diaspora/diaspora which uses MVCP (Presenter)
The other bit it to make use of concerns in Rails 4 for extra DRY controllers and models:
https://stackoverflow.com/questions/14541823/how-to-use-conc...
Well, this is weird. A few hours ago, I tweeted how I dislike Rails::Engine(https://twitter.com/pothibo/status/433595617383047168). I'm going to write a post about it sooner or later but here's a few reasons on top of my head why I dislike them:
- Rails' use of folder for autoloading can encapsulate part of your application like engine does. (app/models/admin/post.rb vs app/models/post.rb)
- Different dependencies will turn on you in the long run. You're shoveling the problem ahead.
- Routing between different engine is cumbersome because of lack of autodiscovery (You can't know easily at run time which engine is mounted)
- Rails' use of folder for autoloading can encapsulate part of your application like engine does. (app/models/admin/post.rb vs app/models/post.rb)
- Different dependencies will turn on you in the long run. You're shoveling the problem ahead.
- Routing between different engine is cumbersome because of lack of autodiscovery (You can't know easily at run time which engine is mounted)
To your points:
- yep. It is unfortunate. Always stick to the default folder structure (subfolders for modules) to prevent any weird interactions among engines and between engines and the main app.
- Prevent this by not allowing this. You would not have different dependencies within one Rails app and you can't have it with a component-based architecture.
- Routing between engines should be much less frequent than routing within. A perfect case for preventing unnecessary dependencies by creating "engine routing intrefaces"
- yep. It is unfortunate. Always stick to the default folder structure (subfolders for modules) to prevent any weird interactions among engines and between engines and the main app.
- Prevent this by not allowing this. You would not have different dependencies within one Rails app and you can't have it with a component-based architecture.
- Routing between engines should be much less frequent than routing within. A perfect case for preventing unnecessary dependencies by creating "engine routing intrefaces"
I would love to read it. I noted and am worried about the dependency thing, though I think that would still happen anytime they were all in the app/bundler regardless of engine use or not.
For "knowing what's enabled" we use the BootInquirer I mention, though we've never had to to know at runtime. The interesting stuff comes up when we have to sync the routing to our load balancer, which we haven't fully automated, though it seems possible.
For "knowing what's enabled" we use the BootInquirer I mention, though we've never had to to know at runtime. The interesting stuff comes up when we have to sync the routing to our load balancer, which we haven't fully automated, though it seems possible.
It was amusing to read through this and just have it slowly confirm all of my experiences with Engines - down to the global spec folder.
It's hella warty in places but overall it provides a lot better "separation of responsibilities", and I too recommend this approach.
It's hella warty in places but overall it provides a lot better "separation of responsibilities", and I too recommend this approach.
I have only ever used the global spec folder as a place to store top-level app integration specs. I think it is a huge mistake to give up on the separation of engine code for tests.
The whole "dummy app just for booting specs" is silly, though. I ran into a ton of trouble trying to get it to play nice, just because my Engine had some dependencies from the main app. Not to mention fighting with rspec.
I agree with you if it's intended to be packaged as a gem, but if I'm only using it as a namespace separator, then it's pointless to fret about where it all goes - they were never meant to be used separately in the first place.
I agree with you if it's intended to be packaged as a gem, but if I'm only using it as a namespace separator, then it's pointless to fret about where it all goes - they were never meant to be used separately in the first place.
I'm completely on the opposite end of the spectrum: engine dummy apps aren't just great for running specs, they're great for running in dev mode (especially when the entire integrated app is a huge slow beast), and for exposing dependencies on the main app, which you ideally want to eliminate. Essentially, the more difficult it is to get your engine running with a plain vanilla dummy app - the more you have to pull in or mock out from the main app - the more tightly coupled they are, and in a gross circular dependency relationship too. If you want the benefits of modularity and separation of concerns, you want the relationship between the engine and the main app as simple as possible; keep the API small and well-defined, and the dependencies going in a single direction.
This is basically our finding as well.
This is a great article! The only thing I would improve is the typography. It was hard to read the entire article because the line length was too long. The font for the text is a bit small. The difference between the headers and the subheaders is not large enough. Since some of you taskrabbiters play here, hopefully you'll take that into account.
While working at a previous start up, I wrote an application platform that allowed Plugin developers to write Client side javascript that was an AJAX bridge to Rails models that auto-generated models, migrations, views, etc. It was a way to allow external developers to write nothing but Javascript in a browser, but write robust IT applications on top of the Rails app that provides relevant data for plugin developers.
Rails Engine seems like a full ruby realization of this previous dream of mine.
Rails Engine seems like a full ruby realization of this previous dream of mine.
I've tried Engines in the past and came away with the feeling that they are just a "one toe in the water" approach to Service oriented Architecture. The hills that you have to climb in regards to testing, assets and dependencies just don't seem worth it. So we created an Engine because we feel that parts of our code base are different enough that they should stand alone, but we then want to share code (a cool way of saying "coupling") between them? It just doesn't add up for me.
The fact that it's a "one toe in the water" approach to SOA is exactly what makes it attractive. It can help you break up the Monorail incrementally without the immediate expensive investment in fully separate apps.
The ease of running integration/acceptance tests in this kind of setup is a huge help, especially if the app already has tests covering the behavior you're trying to maintain while splitting things out.
The ease of running integration/acceptance tests in this kind of setup is a huge help, especially if the app already has tests covering the behavior you're trying to maintain while splitting things out.
They've also changed significantly over the last few years, so if you didn't see them recently, you may want to look again.
As someone who's coming from a Django world and is trying to get into the Rails 4 "way of things," where does this article lie in my path to enlightenment?
Probably towards the end (not that it's necessarily the goal). Make a normal Rails app, and see that grow and the joys and pains that follow. Then see if something like this would make the next one better.
This is amazing. I think the BootInquirer class is going to solve all my problems. I have a large application that is broken into 5 different rails app's and two shared lib gems. I have been looking into using engines but figured I would loose the leanness I got from separating the application.
One very large advantage of multiple apps is they can run on different hardware with separate databases. For example my attachment service runs on a server with libreoffice and x11 installed.
So in theory using the BootInquirer I should be able to boot multiple variations of the application.
For instance I could boot all engines when in development mode but deploy them as separate services with separate databases in production.
Thanks taskrabbit I now know how I am going to spend the rest of my week.
One very large advantage of multiple apps is they can run on different hardware with separate databases. For example my attachment service runs on a server with libreoffice and x11 installed.
So in theory using the BootInquirer I should be able to boot multiple variations of the application.
For instance I could boot all engines when in development mode but deploy them as separate services with separate databases in production.
Thanks taskrabbit I now know how I am going to spend the rest of my week.
Yeah!
BTW, if this kind of thing is interesting to you, TaskRabbit is hiring https://www.taskrabbit.com/careers