Hi HN, Eric here. I did a thing, and I'm super excited to share. By leaning on S3's strong read-after-write consistency, I made a datastore without a persistent database server. As in no binary and no daemon. Just your app and a bucket.
baerly-storage is a document database with no separate database process: the read/write machinery runs to completion inside your request handler, and the durable state lives in S3/R2.
The load-bearing trick: for each collection, a write commits by creating the next numbered log object (`log/<seq>.json`) with `If-None-Match: ""` create-if-absent. That one conditional create is* the commit. `current.json` is not the latest-commit pointer; it is compaction state plus hints. Readers load the snapshot it names and then forward-probe the dense numbered log by GET until the first missing sequence.
If two writers race, exactly one wins a sequence. If a writer sees the slot occupied, it reads the occupant back: same session, same seq, same full entry means "my success response was lost"; anything else means a peer won and this writer retries at the next empty slot. New index markers are written before the log create, stale markers after, so a crash can leave extra index candidates, but should not hide committed rows. Reads stay pure; bounded compaction/GC rides successful writes.
The TypeScript package is the first implementation. The actual contract is the object layout and protocol: content-addressed document bodies, a dense numbered log, snapshot objects, advisory index markers, and `current.json` as compaction state rather than commit authority. Another implementation could speak the same protocol if it honors the same conditional-write and tail-probe rules.
I built this at Gusto because we're seeing a lot of AI-authored internal tools like dashboards, trackers, one-off Claude artifacts that are real enough to need shared durable state but not big enough to justify Postgres + Docker + secrets + on-call and all the ceremony that implies.
The framing that stuck for me was that you need three primitives to build software these days: compute, tokens, and storage. For these workloads, compute has an answer (FaaS, scale to zero), and tokens are just an API call. Storage was missing one.
The cool thing is that this only became possible recently. S3 shipped strong read-after-write in late 2020 and create-if-absent conditional writes in Aug 2024. It's the same "write immutable data, publish with a small atomic step" playbook as Iceberg / Delta / Litestream, but the high-frequency coordination point is the log append itself, so there's no catalog service or lock table.
I did a bunch of conformance and benchmark testing and, TBH, this thing scales better than I expected. Something like 30 requests per sec sustained is still reasonable on both perf and cost (I have a whole writeup in the docs). But, obviously, this has constraints. It gives you per-collection linearizability but no cross-collection transactions or joins, so the best fit test is "can this app's most important screen come from one collection?" It's read-cost-heavy at scale (every read folds a log tail), and at high write volume, the S3 bill from Class A operations starts to price you out.
But also, that's the graduation signal, and I'm considering graduation a success case. I made a CLI to give a clean SQL exit: `baerly export --target=postgres`.
Oh, and as a bonus, the API treats LLMs as a first-class consumer, and part of my test suite was to zero-shot ~7 different kinds of apps with a Sonnet-class model. The whole public API is 8 verbs and 6 predicate operators. All typesafe, with a 12k-token surface that is small enough to fit in context.
Happy to go deep on the consistency model, which was the hard part.
Risk exposure is a BIG issue and is actually one of the primary points of leverage since we were transitioning from a Backbone meta-library that had recently been deprecated. I'll agree that tooling choice needs to be a good technical AND business decision but I do think it's important to address the concerns of those who are maybe gunshy after their last transition from maybe jsp -> jQuery.
I agree that the information is technical but it's not above most many of the business side people who can speak the language a bit. I'm confident that most of these things can either be directly tied to the bottom line or help dispel concerns about how a transition can be made effectively.
Hi acdha - sorry you didn't find this helpful. My intention wasn't really to bring anything new to the table. This actually was an adaptation of the real proposal we used to sell the technical leadership in a switch from Backbone based UI.
Mostly I've found blog posts about react to come in 3 flavors:
1. Something geared towards an audience that lives and breathes web and wants to learn about new tech
2. somebody's experience integrating
3. A tutorial of some kind illustrating how to get started.
It was my hope that I could help provide a concise starting point for others who are trying to migrate old stacks to React not to convey new insights :).
Toy apps teach you how to use libraries and google error messages. Building something will force you to spend time thinking about how not to code yourself into a corner, how to structure your app and how to be efficient. It will also give you an outlet to put all the things you lean in books/railscast etc into practice.
baerly-storage is a document database with no separate database process: the read/write machinery runs to completion inside your request handler, and the durable state lives in S3/R2.
The load-bearing trick: for each collection, a write commits by creating the next numbered log object (`log/<seq>.json`) with `If-None-Match: ""` create-if-absent. That one conditional create is* the commit. `current.json` is not the latest-commit pointer; it is compaction state plus hints. Readers load the snapshot it names and then forward-probe the dense numbered log by GET until the first missing sequence.
If two writers race, exactly one wins a sequence. If a writer sees the slot occupied, it reads the occupant back: same session, same seq, same full entry means "my success response was lost"; anything else means a peer won and this writer retries at the next empty slot. New index markers are written before the log create, stale markers after, so a crash can leave extra index candidates, but should not hide committed rows. Reads stay pure; bounded compaction/GC rides successful writes.
The TypeScript package is the first implementation. The actual contract is the object layout and protocol: content-addressed document bodies, a dense numbered log, snapshot objects, advisory index markers, and `current.json` as compaction state rather than commit authority. Another implementation could speak the same protocol if it honors the same conditional-write and tail-probe rules.
I built this at Gusto because we're seeing a lot of AI-authored internal tools like dashboards, trackers, one-off Claude artifacts that are real enough to need shared durable state but not big enough to justify Postgres + Docker + secrets + on-call and all the ceremony that implies.
The framing that stuck for me was that you need three primitives to build software these days: compute, tokens, and storage. For these workloads, compute has an answer (FaaS, scale to zero), and tokens are just an API call. Storage was missing one.
The cool thing is that this only became possible recently. S3 shipped strong read-after-write in late 2020 and create-if-absent conditional writes in Aug 2024. It's the same "write immutable data, publish with a small atomic step" playbook as Iceberg / Delta / Litestream, but the high-frequency coordination point is the log append itself, so there's no catalog service or lock table.
I did a bunch of conformance and benchmark testing and, TBH, this thing scales better than I expected. Something like 30 requests per sec sustained is still reasonable on both perf and cost (I have a whole writeup in the docs). But, obviously, this has constraints. It gives you per-collection linearizability but no cross-collection transactions or joins, so the best fit test is "can this app's most important screen come from one collection?" It's read-cost-heavy at scale (every read folds a log tail), and at high write volume, the S3 bill from Class A operations starts to price you out.
But also, that's the graduation signal, and I'm considering graduation a success case. I made a CLI to give a clean SQL exit: `baerly export --target=postgres`.
Oh, and as a bonus, the API treats LLMs as a first-class consumer, and part of my test suite was to zero-shot ~7 different kinds of apps with a Sonnet-class model. The whole public API is 8 verbs and 6 predicate operators. All typesafe, with a 12k-token surface that is small enough to fit in context.
Happy to go deep on the consistency model, which was the hard part.