Short summary: Trino uses JDBC in ways in which it was not designed and this can result in massive performance bottlenecks. Sometimes partitioning source data can mitigate the bottleneck.
A lot of people don't realize how slow it is to access data in bulk from a database using JDBC. JDBC was never designed for the use case that Trino uses it for.
This article summarizes research from my lab in collaboration with ByteDance published in CIDR (a computer science conference held in Amsterdam two weeks from now) on a new columnar format designed for ML workloads.
I agree that in theory they could both co-exist for the reasons you state, but in practice I think it's unlikely a company that invests in a data fabric (which is largely a technology cost) is going to simultaneously invest in the incentives for the data product creators that are necessary for the data mesh not to become the wild west.
In general, whenever you need to perform a join (of multiple datasets), that ends the pipeline of local operations on a partition. Other operators as well that necessarily require data from other partitions end local pipelines. This is why linear scalability is not completely achieved in practice. Most interactions with data cannot be performed in a completely partitionable way.
Usually those other operations which force the local pipeline to end occur in a query plan prior to hitting any kind of tradeoff of doing too much in a local pipeline, since local pipelines are SO much faster than what happens when communication is required.
IMO, it's hard to put a price on strong isolation and consistency. Being able to write an app that that uses atomic transactions, that are isolated from concurrently running transactions, and that see the correct data is something that translates to reduced programmer time and effort, and improves user experience. Many programmers discount those important features when they start out, but they'd be better served including them in the price comparisons of different products that are out there.
See what I wrote below regarding Spanner. The same thing applies to the CockroachDB solution. If you run 2PC for multi-region transactions that is very slow (increases latency), and prevents conflicting transactions from running for longer periods of time (decreases throughput).
Let's say you have a transaction that writes data located in different quorums atomically. This is called a "multi-home" transaction in SLOG. Most of the machinery in the paper is dealing with that case. SLOG can do that with half-round trip latency. No idea how long it would take the version of Spanner you described to do that, but presumably at least two full round trips across quorums for two-phase commit.
Latency is measured from the client. In the example in the post (and more details in the paper), you see the latency tail from when clients access data that is far from them. The challenge is to make multi-home transactions no worse than regular Paxos latency. In previous systems, this required multiple rounds of communication across the homes that are involved in that transaction. In systems like PNUTS, they would disable such transactions altogether. SLOG's ability to handle such transactions with latency no worse than Paxos is a big step forward.
As mentioned in the post: "There are several ways to achieve [serializability] — such as via locking, validation, or multi-versioning."
Deadlock happens under some, but not all implementations of serializability via locking. There have been several database systems developed in my lab that use locking to achieve serializability, but yet never deadlock. Examples include:
Bottom line: serializability does not necessarily mean deadlock. Deadlock can be avoided via non-locking implementations, or even in well-designed locking implementations.
One of the points in the conclusion of the post warrants being reiterated at this point:
"If you find that the cost of serializable isolation in your system is prohibitive, you should probably consider using a different database system earlier than you consider settling for a reduced isolation level."
magicalhippo asked a similar question both with regard to complexity and garbage collection. I think it is best to combine these threads, so please see my response there ... https://news.ycombinator.com/item?id=19003212
Every transaction has an numeric identifier. These identifiers are used to order versions of updates to a data item. Versions from higher transaction IDs are considered to be "after" lower ones. Reading versions as of a particular version identifier must be consistent. But the system as a whole needs not be deterministic.
It is one coordinator per transaction. As far as coordinator robustness, I discussed this in the following paragraph from my post:
"There are two categories of work-arounds to the blocking problem. The first category of work-around modifies the core protocol in order to eliminate the blocking problem. Unfortunately, these modifications reduce the performance --- typically by adding an extra round of communication --- and thus are rarely used in practice. The second category keeps the protocol in tact but reduces the probability of the types of coordinator failure than can lead to the blocking program --- for example, by running 2PC over replica consensus protocols and ensuring that important state for the protocol is replicated at all times."