Atomic looks quite interesting, and the "wiki synthesis" is particularly interesting:
I've been working on a suite of skills and a tiny MCP (also SQLite + SQLite-vec based) where the focus is on making it easy to produce "atoms" from quick brain dumps.
The chunking problem is "bypassed" by declaring each section a chunk, and having the LLMs rewrite drafts to sections that chunk well. That means lots of redundancy, and no "As explained above".
The intended reader isn't a human, but rather agents that generate human-friendlier prose, for different target audiences. By assuming the reader is an "expert", the idea is that it's much cheaper to mass-produce reviewed "atoms".
Itching to try that workflow with Atomic or Tolaria.
Or from the other perspective of the trade-off: One caveat with MSSQL is that ALL concurrent transactions must pay the overhead if _some_ transactions need serializable guarantees?
Nice job, eugene-khyst. Looks very comprehensive from an initial skim.
I've worked on something in the same space, with a focus on reliable but flexible synchronization to many consumers, where logical replication gets impractical.
> A long-running transaction in the same database will effectively "pause" all event handlers.
… as the approach is based on the xmin-horizon.
My linked code works with involving the MVCC snapshot's xip_list as well, to avoid this gotcha.
Also, note that when doing a logical restore of a database, you're working with different physical txids, which complicates recovery. (So my approach relies on offsetting the txid and making sure the offset is properly maintained)
Reading that thread it doesn't seem like the official image shipped with any cryptominer at any point, and that it's more likely that the container got compromised in other ways. (A compromised [superuser connection to Postgres can execute shell code](https://medium.com/r3d-buck3t/command-execution-with-postgre...), so that seems more likely than the image shipping with a miner)
Advisory locks are purely in-memory locks, while row locks might ultimately hit disk.
The memory space reserved for locks is finite, so if you were to have workers claim too many queue items simultaneously, you might get "out of memory for locks" errors all over the place.
> Both advisory locks and regular locks are stored in a shared memory pool whose size is defined by the configuration variables max_locks_per_transaction and max_connections. Care must be taken not to exhaust this memory or the server will be unable to grant any locks at all. This imposes an upper limit on the number of advisory locks grantable by the server, typically in the tens to hundreds of thousands depending on how the server is configured.
It’s the wat I’ve seen have the most security impact.
Deep merging two JSON parsed objects is innocuous enough everywhere else that most don’t think twice about doing it. Lots of widely used libraries that provide deep merging utilities have had security vulnerabilities because of this.
I guess you could argue that the wat is that objects coming out of JSON.parse don’t have null as its prototype.
I’d agree that ReDoS is a repeat offender in having overblown severity in vulnerability reports, and prototype pollution reports have contributed to a fair bit of noise when popping up in dev-tools and such, but prototype pollution can be quite significant.
Java has its “gadget chain” class of vulnerabilities, where the presence of certain jars can turn object deserialisation into RCEs. I’d argue that Javascript has “pollution gadgets”.
Some years ago I struggled making lodash – which almost any non-trivially sized Javascript project has at least a transitive dependency on (possibly multiple versions of) – fix its “gadget” in its template function. It’s since been patched, and the conversation unfortunately deleted - https://github.com/lodash/lodash/pull/4518
It has long since been cleaned up, but the history of Lucene's Levenshtein automatons are a fascinating mix of pragmatism, hackery, grit, and engineering. Using a Python library to generate crazy-looking Java code, which you can't (yet) understand, but whose tests pass and benchmarks look amazing.
git's the kind of tool where understanding how your commands operate on the underlying data structures will probably make it a lot easier to use efficiently. (And they're beautifully simple despite how powerful and flexible they are)
As a tool you might be using for hours per week for a few more decades, it's worth the investment going beyond the "in x minutes". I tend to provide both to juniors.
> In this system, the client would request a range (start time, max count) of events via an HTTP request, and the response would include the "end time" that can be used in the next query.
What happens if two transactions commit out of order? tx1 with a lower timestamp commits after tx2 with a higher timestamp has committed - and your client just saw tx2's timestamp.
Or if you have ≥$maxCount number of events changed the same exact timestamp?
> Its was actually very surprising to me to find PG does not cache query plans but instead replans on every execution!
If you used prepared statements, `plan_cache_mode` (since Postgres >=12) can be configured.
It defaults to auto, which will generate a generic plan after 5 (iirc) executions.
If set to `force_generic_plan`, it'll make a generic plan right away. If set to `force_custom_plan`, it'll always do planning. The latter can be useful for queries where the parameters of the prepared statement can have very different selectivities.
I've been working on a suite of skills and a tiny MCP (also SQLite + SQLite-vec based) where the focus is on making it easy to produce "atoms" from quick brain dumps.
The chunking problem is "bypassed" by declaring each section a chunk, and having the LLMs rewrite drafts to sections that chunk well. That means lots of redundancy, and no "As explained above".
The intended reader isn't a human, but rather agents that generate human-friendlier prose, for different target audiences. By assuming the reader is an "expert", the idea is that it's much cheaper to mass-produce reviewed "atoms".
Itching to try that workflow with Atomic or Tolaria.