Thanks, this is neat. Goodreads seems to be get so little development attention these days that it's a miracle it still runs.
Interesting choice by BookWyrm to hide review ratings by default. I like it — I always felt that on Goodreads literature that's a little more challenging to tackle is reviewed very unfairly compared to easy-to-digest-but-content-light popular books or pulp fantasy. Making the default to have users read the contents of a review instead of glancing at an aggregate number seems like the right answer.
It's true that it's not distributed, but there are a lot of benefits to not going distributed immediately, like extremely predictable data consistency. I would hazard to guess that the _vast_ majority of apps that are not built by the superscalers are already using a database like Postgres or SQLite to store their data, and River merely suggests that you hook your job queue into the database that you already have.
There's some similarities with DBOS for sure like using Postgres as a backend. Here's some differences based on my browsing of their docs for a little while:
* River's built around an entirely open core with its bread and butter being background jobs rather than workflows, with basic background jobs being good enough for most apps in most situations. It's true that workflows are gated behind Pro, but a lot of users will find they won't even need them, or won't need them until much later.
* River's aimed more solidly at Go, especially for the running of the background jobs themselves. Blake and I are both experienced Go developers, and we've gone through great pains to make the API as elegant as possible and as easy-to-use as possible, aiming for things like consistency and predictable + well-documented APIs. DBOS supports Go as well, but I believe our API compares very favorably [1], though you can be the judge.
* I might be missing something in the DBOS docs, but especially pertaining to background jobs, I believe River's feature set is quite a lot more comprehensive. e.g. Bulk insertion, unique jobs, periodic/cron jobs, job snoozing, job scheduling, unique jobs, test helpers, etc. We've tried to include everything that people would need when building out with background jobs, including all the edge cases.
Lastly, to be fair, DBOS is price-gated as well [2], and pricing is based on usage whereas River's is not.
Yes, I roughly agree with all of this. In fact, for most of my existence, I'm been one of those cheap programmers.
The circumstances that led to me trying to push River for the next few months were somewhat accidental, and it felt like a good moment to at least make a go of trying to make it work. I'm not committing the rest of my career/life to any particular decision one way or the other.
I'll reiterate too that I believe we're still quite early in the LLM age and are still waiting for the other shoe to drop. All LLM-generated software feels free at the moment because it's still novel and the exhilaration of accomplishment when you build something complex inside of a few hours is addictive beyond words. However, within a year or two I think we're going to have a lot more software, all of which needs maintaining to some degree, and we're going to become a little more reluctant to generate new projects to add to the heap. This'll cause an adjustment back to a more compromise position.
(Also, could be completely wrong about all of that, so take it for what it is.)
To be fair I guess, the way the article is titled goes out of its way to mislead.
It would be quite easy to say "Added 911,000 fewer jobs from March 2024 to March 2025" or "the year starting in March 2024", but they are clearly aiming to deflect from the Biden admin by implying last year's revisions are the fault of the administration inaugurated in January 2025.
Judging by the comments here, it worked marvelously.
That's absolutely bananas. $3 million for a _report_ about suggestions for possible immigration reform (to speed it up of course). In the hole $62 billion a year, and a sizable chunk is going to overpriced MBA grads right out of school to produce PDF documents. How the hell do we reign these people in.
Are you my doppelganger? I made almost this exact comment word-for-word to a friend of mine a few weeks ago.
Trudeau's immigration video from December [1] was one of the most dishonest, condescending productions that I've ever seen, basically amounting to, "Yes, we destroyed our previous internationally respected immigration system and imported five million low skilled laborers over a couple years without adding any housing or infrastructure. Yes, that's hurt a lot of you and made you angry. No, we don't think that's a problem. But because you're so angsty, we'll throttle it back a tiny wittle bit over the next year or two before throwing the floodgates wide open again."
Not a shred of anything even resembling self-awareness or humility throughout.
The point is that Democratic voters didn't get a chance to have their voice heard. Conducting polling is post-hoc rationalization for Harris being installed by party leaders in unprecedentedly anti-democratic action.
Also notable that the author didn't link Haje's (the TechCrunch person who railed against Alexander Wang) actual writing because he knew it doesn't make either Haje, or his argument in favor of Haje, look good. So, here it is:
> I would invite him — and those supporting them — to fuck all the way off. You misunderstand me. You thought I wanted you to fuck only partially the way off. Please, read my lips. I was perfectly clear: Off you fuck. All the way. Remove head from ignorant ass, then fuck all the way off.
This is the quality of writing whose loss the author is arguing is why TechCrunch has lost its global relevance. A perfect inversion of the truth if I've ever seen one.
I found this by accident on YouTube the other day, and started watching it, not intending at all to sit through the 4+ hour runtime, but by the next day I'd watched the entire thing.
I'm not a Disney Parks person at all, but found the whole thing fascinating. Honestly, at first glance, this seems like such a fun idea, and given the popularity of other parks projects, I couldn't believe it'd failed. But digging in, it all kind of starts to fit together.
I didn't take notes, but off the top of my head:
* The stay was very expensive. You actually had to talk to an agent for specific pricing, but it was around $4k USD for two people for two days, with additional charges if you wanted to add people to your room.
* Despite luxury pricing, the rooms and furnishings are decidedly not. Tiny cabin-style rooms with bunk and pull out beds, and ~no amenities (no gym, no pool, etc.).
* The experience seems to have been a good idea, but executed poorly. This YT creator had trouble accessing any of the theoretically available storylines, and despite an effort to interact as much as possible ended up with results not discernible from random, or having done nothing at all.
* Lots of boring app-driven interactions involving messaging with virtual avatars of the IRL characters. On the day excursion to the rest of the Star Wars world, lots of of scanning QR codes on crates. The whole thing seems to have been very uninspired, and buggy to boot.
* When comparing the final product to concept art and statements from Disney execs, it seems like a lot of corners were cut for the final product and it ended up very underwhelming compared to what it was supposed to be. There seems to be lots of evidence elsewhere that Disney parks have really become cheapskate / penny pinching operations, and a reasonable hypothesis is that there were a lot of big ideas, but they would've been expensive, and they were slowly cut back one by one until the overall product really had nothing special left, and fell flat.
* From Disney's end, it didn't seem to scale very well. The high price tags seems to be related to the fact that the hotel didn't have all that many rooms compared to larger resorts which might have thousands, so they felt that they had to recoup the cost from each one. This didn't sit well with customers though, and after the initial opening period, the hotel seemed to be operating at high vacancy rate because few people were able/willing to pay the very substantial premium.
* Perhaps the biggest reason for its abrupt closure is due to accountants. Disney will be taking up to a $300 million tax write off from the project, so its closure may have been even a neutral or even good thing to its execs. The closure was extremely abrupt, which seems to suggest that they wanted to take the write off for the prior fiscal year specifically.
The reason you'd use a non-blocking send is to make sure that in the event of one slow consumer that the entire system doesn't slow down.
Imagine a scaled out version of the notifier in which it's listening on hundreds of topics and receiving thousands of notifications. Each notification is received one-by-one using something like Pgx's `ListenForNotification`, and then distributed via channel to subscriptions that were listening for it.
In the case of a blocking send without `default`, one slow consumer that was taking too much time to receive and process its notifications would cause a build up of all other notifications the notifier's supposed to send, so one bad actor would have the effect of degrading the time-to-receive for all listening components.
With buffered channels, a poorly written consumer could still drop messages for itself, which isn't optimal (it should be fixed), but all other consumers will still receive theirs promptly. Overall preferable to the alternative.
In Postgres listen/notify are inherently lossy channels — if a notification goes out while a listener wasn't around to receive it, it's gone, so they should never be relied upon in cases where data consistency is at stake.
I find that the main thing they're useful for is notifying on particular changes so that components that care about them can decrease the time until they process those changes, and without sitting in a hot loop constantly polling tables.
For example, I wrote a piece here [1] describe how we use the notifier to listen for feature flag changes so that each running program can update its flag cache. Those programs could be sitting in loops reloading flags once a second looking for changes, but it's wasteful and puts unnecessary load on the database. Instead, each listens for notifications indicating that some flag state changed, then reloads its flag cache. They also reload every X seconds so that some periodic synchronization happens in case an update notification was missed (e.g. a notifier temporarily dropped offline).
Job queues are another example. You'll still be using `SKIP LOCKED` to select jobs to work, but listen/notify makes it faster to find out that a new job became available.
Author here. The Go channel send behavior could certainly be altered depending on the particular semantics of the application, but the reason I chose to use a non-blocking buffered channel is so that no particular subcomponent can slow down the distribution of notifications for everybody.
> Shouldn't the channel rather block than discard if full?
In Go, a blocking channel is one that's initialized without a size (see [1]). You could have a blocking channel where the sender uses a `select/default` to discard after it's full, but that leaves very little margin of error for the receiver. If it's still processing message 1, and then message 2 comes in and the notifier tries to send it, message 2 is gone.
IMO, better to use a buffered channel with some leeway in terms of size, and then write receivers in such a way that they clear incoming messages as soon as possible. i.e. If messages are expected to take time to process, the receiver spins up a goroutine to do so, or has another internal queue of its own where they're placed so that new messages from the notifier never get dropped.
Author here. The behavior of notify with respect to transactions is indeed notable, and definitely a great feature that makes them distinct from pub/sub in other systems. Notifies fire only when data is ready after the transaction is committed, and they're also deduplicated based on payload so listeners don't have to react to many of the same message unnecessarily.
That said, NOTIFY isn't really what this post is about. It concerns itself with the other half of listen/notify by describing a "notifier" pattern, one which listens via `LISTEN` statements and distributes them application subcomponents to help maximize economy around the use of Postgres connections.
Listen does hold a connection, but that doesn't mean it defeats connection pooling.
That's what I was trying to convey in this blog post: you'll keep a fixed number of connections open for use with listen, but as few as possible by reusing a single connection per program to simultaneously listen on all channels that your application cares about (with the notifier distributing messages to each internal component that subscribed).
With your dedicated listen connections accounted for, the rest of the connection pool can operate normally, with programs checking connections in and out only as long as they need them.
So the net-net is that you have a handful of connections dedicated for listen, and the remaining ~hundreds are part of the connection pool for shared use.
That's interesting — the typed request and response objects that you get with an API SDK are certainly one of the biggest upsides, and I certainly see the benefit of having full control of the transport layer yourself so that you can add any common telemetry / logging / statistics at your leisure.
On the other hand, a benefit of a more complete API is that in typed languages you can tab-complete your way to success. With each endpoint a function and all the requisite configuration (API URL, etc.) bundled in, for basic integrations you may never even have to reference documentation, or if you do, very little of it, as your IDE finds functions/properties for you and you can read documentation right out their docstrings.
I helped maintain the public API language bindings at Stripe for many years, and although I'd defend it to the death as the right DX (SDKs are so much easier to use than raw web APIs), it's hard to appreciate just how onerous of a process this was before Alex pioneered a codegen-based solution.
Some of the language SDKs were dynamic (e.g. Ruby, Python) and could adapt automatically to small API changes like if a new field was added in a response. Some were not (e.g. Java, Go), and for every API change, someone had to manually make the change to that SDK's codebase (add a field, add a struct, add a function for a new endpoint), get it reviewed and merged, and cut a release. As Stripe got bigger and there were API changes all the time, the only way this was even remotely functional was that we had a couple heroic workhorses that'd watch for changes and open hundreds of PRs a year for them. Honestly, in retrospect, it's amazing this even worked.
Getting everything switched over to a generated solution was an arduous process because the new generated code had to be API-compatible with the existing code so that the cutover to generated bindings didn't break every Stripe user under the sun. We eventually got there, but it took a long time.
I like Stainless' mission because after seeing the crazy maintenance hassle that all of this was at Stripe, I think it makes way more sense to save all that engineering time for your more concerns, and outsource this problem to someone else. A plug-and-play way of getting high quality SDKs in all common languages that get pushed to appropriate repositories for distribution and comes with quality companion documentation.
We've actually had pretty good luck at my current job with open source tools like openapi-generator [1], but this sort of codegen is so convoluted that the sustainability of a pure open source solution makes me a little afraid, and you still end up doing a lot of the last mile work yourself.
Twitter: https://twitter.com/brandur
GitHub: https://github.com/brandur
Website: https://brandur.org
Email: [email protected]