I always tell people there are two clear areas where DynamoDB has some major benefits:
- Very high scale applications that can be tough for an RDBMS to handle
- Serverless applications (e.g. w/ AWS Lambda) due to how the connection model (and other factors) work better with that model.
Then, for about 80% of OLTP applications, you can choose either DynamoDB or RDBMS, and it really comes down to which tradeoffs you prefer.
DynamoDB will give you consistent, predictable performance basically forever, and there's not the long-term maintenance drag of tuning your database as your usage grows. The downside, as others have mentioned, is more planning upfront and some loss of flexibility.
One difference with DynamoDB is that there's no query planner, so you can have a pretty good sense of how many items you'll hit and how big that read is.
Question for you -- what are the performance implications of "re-keying" my records during the transform? Should I try to keep the same key for my transformed record as from my original record so that they align with the same partitions, or are they likely going to get sent to different partitions across different brokers anyway?
Wow! Congrats to you, Jay and Frank. I've been a fan of your work on both Seed.run & Serverless Stack for a while. Best of luck, and I'm excited to see Seed grow :)
I highly recommend this book and Swyx's other work. He's thorough while also distilling down to the most important bits.
Great book for people that are just starting their coding career but also for those with a good bit of experience. If you're feeling 'stuck', this is a great guide to understand how to advance your career.
'Learn in public' is probably the #1 piece of advice I'd give to anyone in tech. It helps your writing, it builds your network, and it grinds down your ego (because people will certainly let you know when you're wrong). Swyx has been a huge proponent of this, and this whole book is a great kick in the pants to get started.
I'd be shocked if this book doesn't make you >>>10x the amount you spend on it (even counting a healthy hourly rate for reading it).
For me, I like that 98% of DynamoDB work is frontloaded. I spend the time building the model but once it's done -- set it and forget it.
With RDBMS, it's like there's a hidden 5% tax that's lurking at all times. You have to spend time tuning querying, reshaping data, changing patterns, etc. It can add up to significant drag over time.
Different teams might think the costs are different for their application, or they may be fine with one pattern over the other. Fine with me! I just know which one I choose now :)
Basically, most of these issues are gone. As long as you don't have extreme skew in your partition keys, you don't need to worry about throughput limits.
I recommend On-Demand pricing 'until it hurts'[0], but that's because a ton of people I talk to are spending <$50/month on DynamoDB. At that point, it really doesn't make sense to spend hours of time optimizing your DynamoDB bill.
If you are at the point where you are spending over thousands of dollars a month on DynamoDB, then it does make sense to review your usage, fine-tune your capacity, set up auto-scaling, buy reserved capacity, etc. But don't waste your time doing that to save $14 a month. There are better things to do.
But it's really nice to have a database where you can set up pay-per-use, don't have to think about exhausting your resources, and have an option to back out into a cheaper billing mode if it does get expensive.
[0] - Hat tip to Jared Short for this advice & phrase
Daniel, I'm a big fan of yours but disagree with this take :).
It's definitely a database. The modeling principles are different, and you won't get some of the niceties you get with a RDBMS, but it still allows for flexible querying and more.
All the examples are specific to DynamoDB and use DynamoDB features.
That said, the principles apply pretty well to other popular NoSQL databases, especially MongoDB and Cassandra. There will be some slight differences -- MongoDB allows better nesting and querying on nested objects -- but it's broadly the same. If you want to model NoSQL for scale, you need to use these general patterns.
If you want to check it out but find out it doesn't work for you, just let me know. I've got a 100% money-back guarantee with no questions asked if you don't like it.
True! I'm not a huge fan of Aurora Serverless and the Data API. The scaling for Aurora Serverless is slow enough that it's not really serverless, IMO. And the Data API adds a good bit of latency and has a non-standard request & response format, so it's hard to use with existing libraries. But it's definitely an option for those that want Lambda + RDBMS.
The RDS Proxy is _hopefully_ a better option in this regard but still early.
Good point! I would usually not recommend using a database from a different cloud provider just because of different hassles around permissions, connections, etc.
I've never found the speed an issue, but YMMV. To me, the best thing is that you won't see speed degradation as you scale. With a relational database, your joins will get slower and slower as the size of your database grows. With DynamoDB, it's basically the same at 1GB as it is at 10TB.
I can understand the sentiment and don't fault you for it.
That said, I think you can definitely handle complex, relational patterns in DynamoDB pretty easily. It will take some work to learn new modeling patterns, but it's absolutely doable.
The biggest advice I can give you is not about any specific tool, it's about an approach. You need to think about how you will market the book if you're self-publishing.
Engage with the community that will be interested in the book. Write articles, help out on Twitter, write code libraries, etc.
For me, I wrote DynamoDBGuide.com two and a half years ago over Christmas break. I wanted to just make an easier introduction to DynamoDB after I watched Rick Houlihan's talk at re:Invent (which is awesome).
That led to other opportunities and to me being seen as an 'expert' (even when I wasn't!). I got more questions and spent more time on DynamoDB to the point where I started to know more. I gave a few talks, etc.
I finally decided to do a book and set up a landing page and mailing list. I basically followed the playbook that Adam Wathan described for his first book launch.[0] Write in public, release sample chapters, engage with people, etc.
In terms of tooling, I used AsciiDoc to generate the book and Gumroad to sell. On a 1-10 scale, I'd give AsciiDoc a 5 and Gumroad an 8. But the tooling barely matters -- think about how to find the people that are interested :)
Happy to answer any other questions, either in public or via email.
- Very high scale applications that can be tough for an RDBMS to handle
- Serverless applications (e.g. w/ AWS Lambda) due to how the connection model (and other factors) work better with that model.
Then, for about 80% of OLTP applications, you can choose either DynamoDB or RDBMS, and it really comes down to which tradeoffs you prefer.
DynamoDB will give you consistent, predictable performance basically forever, and there's not the long-term maintenance drag of tuning your database as your usage grows. The downside, as others have mentioned, is more planning upfront and some loss of flexibility.