Database Performance Optimization and Scaling in Rails(blog.appsignal.com)
blog.appsignal.com
Database Performance Optimization and Scaling in Rails
https://blog.appsignal.com/2022/12/07/database-performance-optimization-and-scaling-in-rails.html
5 comments
Another trap people fall into is doing joins in their application instead of in the database. I see this all the time where people will query some table with a condition, map the ids from the result and then query another table (which also joins the first table) and then filters by the id of the first query. Or just doing massive aggregates in ruby instead of having the database do it and just spitting out the result.
A little bit of knowledge about SQL and how a relational database works goes a long way.
A little bit of knowledge about SQL and how a relational database works goes a long way.
I would say lack of appropriate indexing can be more important.
Adding an index for a query can take it from seconds to me for the average response.
Adding an index for a query can take it from seconds to me for the average response.
It also wraps every query in an explicit transaction unnecessarily causing locks to be held while waiting for the commit from the server which got held up behind some cpu intensive block in separate thread (prob some other request deserializing and hydrating those 100s of entities you mentioned).
Side note: I really dislike Rails.
Side note: I really dislike Rails.
> It also wraps every query in an explicit transaction unnecessarily causing locks to be held while waiting for the commit from the server
no it doesn't. It only wraps updates in transactions, and a single line update would have an implicit transaction at the database anyway.
no it doesn't. It only wraps updates in transactions, and a single line update would have an implicit transaction at the database anyway.
It would have an implicit transaction, wouldn't it?
Think about what we both wrote here (sans the weak man argument, of course I'm talking about updates).
Think about what we both wrote here (sans the weak man argument, of course I'm talking about updates).
I see. Well you could have been more explicit in your original message, I don't think it was obvious at all that you were talking about updates.
Rails does the right thing here in my opinion. Anything that happens between calling `.save` and it returning should be in the safe envelop of a transaction. It can't know that you're only going to emit a single UPDATE query in that transaction, plus the framework user expects any exception between calling save and it returning to roll back. There are outs if you need them. I've worked on a few reasonably high traffic and complex rails applications and I don't recall a time when the time between the last query and the commit caused an issue. Yes times when the transaction itself had got too large and that commit was taking awhile. That said, there were more of these types of locking issues in the project I worked on 10 years ago with MySQL and repeatable read isolation level than with PG since then which defaults to read committed.
Rails does the right thing here in my opinion. Anything that happens between calling `.save` and it returning should be in the safe envelop of a transaction. It can't know that you're only going to emit a single UPDATE query in that transaction, plus the framework user expects any exception between calling save and it returning to roll back. There are outs if you need them. I've worked on a few reasonably high traffic and complex rails applications and I don't recall a time when the time between the last query and the commit caused an issue. Yes times when the transaction itself had got too large and that commit was taking awhile. That said, there were more of these types of locking issues in the project I worked on 10 years ago with MySQL and repeatable read isolation level than with PG since then which defaults to read committed.
from my experience, every performance issue is the database when your programming language makes better choices.
There is so much stuff happening in the ruby on rails.
I follow the developments on YJIT and hotwire and absolutely love the communities there. The design is neat and that gives me confidence the stuff will be maintainable for decades to come.
I follow the developments on YJIT and hotwire and absolutely love the communities there. The design is neat and that gives me confidence the stuff will be maintainable for decades to come.
It goes without saying that there are a lot of things one would do prior to some of these steps. Because of the complexity these sort of configuration bring into your app one should weigh carefully whether they need these approaches. Database tuning and caching are preferable first steps in an app faced with performance challenges before fancy database routing. As with caching the app awareness of these approaches leaks into your code and can introduce testing challenges and misses in your process.
Maybe I missed it but I’m surprised they didn’t mention Multi-Tenancy with Multi-Database since they have otherwise written about it*. For many apps this is not only a way to keep data isolated but is also a way to mitigate performance challenges that might come with a single large DB.
https://blog.appsignal.com/2020/12/02/building-a-multi-tenan...
Maybe I missed it but I’m surprised they didn’t mention Multi-Tenancy with Multi-Database since they have otherwise written about it*. For many apps this is not only a way to keep data isolated but is also a way to mitigate performance challenges that might come with a single large DB.
https://blog.appsignal.com/2020/12/02/building-a-multi-tenan...
I'm impressed how Rails seems not to become obsolete.
Java, PHP, RoR, C, they're all here to stay. Even Perl is still kicking, and it'll be decades still before the last line of COBOL has been written.
I think there is a general misconception that if a language is not new and growing, then it is somehow going away. If anything, it's probably the opposite. Something like inertia is a much better predictor for language lifetime than than growth or features. In practice, the longer a language has been around and the more likely it is to stick around.
I think there is a general misconception that if a language is not new and growing, then it is somehow going away. If anything, it's probably the opposite. Something like inertia is a much better predictor for language lifetime than than growth or features. In practice, the longer a language has been around and the more likely it is to stick around.
> In practice, the longer a language has been around and the more likely it is to stick around.
I think this is called the Lindy effect: https://en.wikipedia.org/wiki/Lindy_effect
> The Lindy effect is a theorized phenomenon by which the future life expectancy of some non-perishable things, like a technology or an idea, is proportional to their current age. Thus, the Lindy effect proposes the longer a period something has survived to exist or be used in the present, the longer its remaining life expectancy.
I think this is called the Lindy effect: https://en.wikipedia.org/wiki/Lindy_effect
> The Lindy effect is a theorized phenomenon by which the future life expectancy of some non-perishable things, like a technology or an idea, is proportional to their current age. Thus, the Lindy effect proposes the longer a period something has survived to exist or be used in the present, the longer its remaining life expectancy.
There's a lot of selection bias wrt who submits posts to HN and the types of content they submit. It creates a sort of perception that tech/libraries/languages are moving near the speed of light and anyone who isn't treading along the bleeding edge is obsolete.
As someone who doesn't write Ruby, even I can grok RoR in a few hours. And even though it's boring and not trendy, it gets the job done for a lot of use cases.
As someone who doesn't write Ruby, even I can grok RoR in a few hours. And even though it's boring and not trendy, it gets the job done for a lot of use cases.
There’s always going to be a demand for pure productivity.
Rails is fantastic! Especially with the new Hotwire/Turbo way of doing things. On a new personal project so far I haven't had to so much as TOUCH front-end JS!
It is alive and kicking. Open Core Ventures had to decide on a tech stack and I think Rails is still the best option https://boards.greenhouse.io/opencoreventures/jobs/475195700...
Boring technologies is just "boring, not obsolete.
Nothing matches it to this day really.
What would have replaced it?
What kind of services do you like to manage all of these "optimizations"? Most developers would get into more trouble trying to setup something like is talked about here, so I assume most people's database provider helps?
Typically, the performance issues are at the ActiveRecord level, when it's initialization 100s of instances of an object to render a table. With Rails, the #1 optimization technique is to use `ActiveRecord::Base.connection.exec_query ` to return a hash instead of initialization objects.
Side note: I still love Rails.