I made this tool so that you can have a pipeline that looks like the following:
planner -> implementer -> reviewer
Running for a number of iterations/loops, where it keeps building like a perpetual motion machine.
I used it for 4 hours to build this to 80% - https://github.com/mj1618/ruckus
Before I dropped down into prompt mode and added some features that I wanted (e.g. themes).
But essentially you can just set this pipeline up with a plan and let it run for days on-end building you a product that you want.
Of course it always depends, but my experience is people reach for distributed systems when they can simply run a single node.
If you can run a single node, and it meets all your business requirements, then I’d argue that is simpler than having to manage the very challenging problems inherent in distributed systems.
A simple web app can run 100k+ connections per second with latency in the 100s of microseconds. This is 2 orders of magnitude better than most apps need.
If you want durability you can use a service like RDS which takes care of replication and backups for you.
I wrote a blog post on this. We were using event sourcing and went with locking+batching method described. For other tables we wanted to read and move to a data warehouse we used the txid check.
A bit hacky, but on AWS RDS we didn’t want to deal with wal intricacies.
Always found the first method of keeping bookmarks to the positions in a table to be the best method. I don’t think the 3 problems given to be an issue personally.
- just loop at the interval that your users can accept the delay
- querying a database 10 times a second is nothing, even 100 times, this wouldn’t cause resource limitations
- the scaling concern made no sense to me, seems a bit arbitrary.
Cool to know this is an option though, but I much prefer not relying on database internals like wal logs, this hurts scaling more IMO.
The only thing you need to worry about with the table method is out of order increment IDs, which is always possible in a transactional database. But there are many solutions for this.
Running for a number of iterations/loops, where it keeps building like a perpetual motion machine.
I used it for 4 hours to build this to 80% - https://github.com/mj1618/ruckus Before I dropped down into prompt mode and added some features that I wanted (e.g. themes). But essentially you can just set this pipeline up with a plan and let it run for days on-end building you a product that you want.
A desktop application is in-bound.