Don't add a read replica until you've read this(incident.io)

14 pointsby shintoist7 dni temu6 comments

Don't add a read replica until you've read this

https://incident.io/blog/dont-add-a-read-replica-until-youve-read-this

4 comments

I work with Johanna (author) on the incident team. It’s been pretty cool watching this roll out, been particularly great that I’ve barely had to think about the replicas when working with the code.

The wiring to route to either replica or primary is very transparent. Seen this done several times before and it’s never been quite as easy to have your code just do the Right Thing.

mbfisher7 dni temu

Also work with Johanna and agree this was fun to work on!

Biggest win for us now IMO is that we can just horizontally scale reads. Instead of trying to play whackamole with micro-optimisiations, we just buy more computers. (Optimisation still has its place obvs!) And, the way this is set up, load balancing across multiple replicas can also be completely hidden from callers.

Neat

mnbbrown7 dni temu

You guys sound like cool guys.

rorymalcolm7 dni temu

Disclaimer: I worked with Johanna on this

I had a lot of fun “long tail whackamoling” our way through edge queries that our model of “this is eligible for a read replica” wasn’t accurately catching. Honestly just one of those “keep reading traces and working through them” development processes that just keeps drive a graph down and to the right.

joehart7 dni temu

Its a shame they didn't go with a Bladerunner replicant pun for the title. "I've seen database performance you wouldn't believe"

patrickhamann7 dni temu

Disclosure: I work at incident.io with the author. My favourite part of how we implemented this is that consistency stopped being something engineers had to continuously think about and became part of the Go context object in our codebase; so any subsequent write in the same context taints without them having to think, and tainted reads go to the primary.