We will eventually add the SERIALIZABLE isolation level to OrioleDB, but right now that's not our highest priority. Let me explain why. At first, SSI (serializable snapshot isolation) in PostgreSQL comes with significant shortcomings, including.
1) Overhead. SSI implies a heavyweight lock on any involved index page or heap tuple (even for reads). The overhead of SSI was initially measured at ~10%, but nowadays, scalability has gone much farther. These many HW-locks could slow down in multiple times a typical workload on a multicore machine.
2) SSI needs the user to be able to repeat any transaction due to serialization failure. Even a read-only transaction needs to be DEFERRABLE or might be aborted due to serialization failure (it might "saw impossible things" and needs a retry).
In contrast, typically it's not hard to resolve the concurrency problems of writing transactions using explicit row-level and advisory locks, while REPEATABLE READ is enough for reporting. Frankly speaking, during my whole career, I didn't see a single case where SERIALIZABLE isolation level was justified.
This is generally true, but there are some additional aspects.
1. With PostgreSQL heap, you need to access the heap page itself. And it's not for free. It goes all through the buffer manager and other related components.
According to all of the above, I believe OrioleDB still wins in the case of secondary key lookup. I think this is indirectly confirmed by the results of the TPC-C benchmark, which contains quite a lot of log of secondary key lookups. However, this subject is worth dedicated benchmarking in the future.
Hey HN, I'm the creator of OrioleDB, an extension for PostgreSQL that serves as a drop-in replacement for the default Heap storage engine. It is designed to address scalability bottlenecks in PostgreSQL's buffer manager and reduce the WAL, enabling better utilization of modern multi-core CPUs and high‑performance storage systems. We are getting closer to GA.
This blog post is dedicated to describing a new OrioleDB optimization: ordered insert optimization. When multiple processes are trying to insert into the same page, the first one that grabs the lock does the job for the others. That dramatically cuts the locker waiting time and thus improves the overall throughput. This is very critical for IoT, time series and others.
We would love for more people to test and benchmark OrioleDB. You can check how ordered insertion optimization would help your workload. The fastest way to do that is to use the Docker image provided (please note you need to use a specific tag):
docker run -d --name orioledb -p 5432:5432 orioledb/orioledb:ordered-inserts-pg17
Hey HN,
I'm the creator of OrioleDB, an extension for PostgreSQL that serves as a drop-in replacement for the default Heap storage engine. It is designed to address scalability bottlenecks in PostgreSQL's buffer manager and reduce the WAL, enabling better utilization of modern multi-core CPUs and high‑performance storage systems.
We are getting closer to GA.
This blog post is dedicated to describing a new OrioleDB optimization: fastpath search, an inter-page search that bypasses page copying and tuple deformation for certain fixed-length datatypes.
We would love for more people to test and benchmark OrioleDB. You can check how fastpath search would help your workload. The fastest way to do that is to use the Docker image provided:
docker run -d --name orioledb -p 5432:5432 orioledb/orioledb
Additionally, OrioleDB beta12 features a new fastpath tree search, which can accelerate workloads with intensive key-value lookups by up to 20%. Stay tuned for a new blog post about this later this week.
> Additionally, PostgreSQL requires N^2 of memory depending on the number of connections,
For sure, not all the PostgreSQL memory is N^2. AFAIR, just a couple of components, including deadlock decoding, require a quadratic amount of memory. Normally, they are insignificant but growing fast if you are rising max_connections.
Thank you for your feedback!
We tried to enable think time with go-tpc, thanks to @pashkinelfe.
That leaves us with 1 tpmC per connection, growing linearly up to ~300 connections for both heap and OrioleDB. So, in order to experience a storage bottleneck, we would need dozens of thousands of connections. Given PostgreSQL runs a process per connection, that would be more of a stress test for the Linux kernel. Additionally, PostgreSQL requires N^2 of memory depending on the number of connections, and it becomes sensible at this scale. All of that could be resolved by migrating PostgreSQL to co-routines and resolving memory requirement issues. However, this is currently out of scope for us.
Could you recommend another benchmark to reveal storage bottlenecks, given that TPC-B and YCSB are too trivial?
The important design issue about building active-active multi-master on the base of raft protocol is about being able to apply changes locally without immediately putting them into a log (without sacrificing durability). MySQL implements a binlog, which is separate from a log, to ensure durability. OrioleDB implements copy-on-write checkpoints and row-level WAL. That gives us a chance to implement MM and durability using a single log.
Some notable benchmarks from the OrioleDB beta7 release:
* 5.5x Faster at 500 Warehouses: In TPC-C benchmarks with 500 warehouses, OrioleDB outperformed PostgreSQL's default heap tables by 5.5 times. This highlights significant gains in workloads that stress shared memory cache bottlenecks.
* 2.7x Faster at 1000 Warehouses: Even when the data doesn't fit into the OS memory cache (at 1000 warehouses), OrioleDB was 2.7 times faster. Its index-organized tables improve data locality, reducing disk I/O and boosting performance.
Run your own workloads or existing benchmarks like go-tpc or HammerDB to see the performance differences firsthand. We Would love to hear about others' experiences with OrioleDB, especially in production-like environments or with different workloads.
This was driven by Andres Freund for pg12. Please, check this.
https://anarazel.de/talks/2018-10-25-pgconfeu-pluggable-stor...
However, the current table AM API in many aspects assumes heap-like storage. This is why OrioleDB comes with a patchset we're intended to upstream to PostgreSQL.
Sure, there are downsides.
1. The replay of WAL records becomes somewhat more CPU-expensive. But at the same time, OrioleDB can do that in parallel.
2. Replace becomes not the file-level equivalent of primary. Thus, you, for instance, can't continue file-level incremental backup started taken from the primary with the replica.
Nevertheless, I strongly believe that row-level WAL is a great win on average, and opens brilliant perspectives for further developments.