Trino is a nice query engine solution if you want to not just run SQL on Elasticsearch but also want to be able to join data from Elasticsearch with data in other systems Trino supports. It also supports raw elasticsearch queries that are serialized back into Trino data types.
Check out Trino Summit 2022 to learn more about the federated query engine and "Federate 'em all". Talks include Lyft, Shopify, Astronomer, Starburst, and much more to be announced in the coming weeks!
Trino (formerly PrestoSQL) is a query engine that originally aimed to replace the Hive runtime and grew into a powerful federated query engine. It can connect to Snowflake, BigQuery, Hive, Oracle, Mongo, Iceberg, Elasticsearch, and many more. This enables powerful joins across your platform from one location.
In this episode of Trino Community Broadcast, we interview Ryan Blue to discuss the latest innovations in Apache Iceberg, why choose Apache Iceberg, latest Trino innovations and much more.
Yeah, though a lot of Fivetran customers are likely the type that would go all in on paying for a conventional data warehouse where people using open source stacks may be the ones that are using open ingestion alternatives.
We see a pretty even mix from the Trino/Starburst lens. Bigger companies like to mix and match.
All use the Lyft "Presto but really Trino"-Gateway project to run different clusters to handle various workloads. They go into various details for how this is achieved.
Regarding the Trino/Presto split. I recommend looking at this blog to better understand why these two communities aren't mergeing. TL;DR Presto is a Facebook-driven project that mainly considers running on the Facebook infrastructure. Trino is community-driven that works on running well with all clouds and common infasturcture in the Trino community which is why you see a higher velocity there.
Clickhouse is a realtime system where Trino is a batch-oriented system. There are tradeoffs for doing realtime vs batch.
Realtime is generally more expensive to run as you process every individual row as it comes, batch is when you can deal with minute latency and want to handle a lot of data in chunks.
It also happens to connect to Clickhouse and it's very common that people will use Trino to query clickhouse realtime data and join it with data in big query, an object store data lake, or Snowflake: https://trino.io/docs/current/connector/clickhouse.html
I mean, the hive migration path is one thing. Now that Iceberg is taking over the old Hive model, data lakes are all the rage again.
The other thing I would say is that Trino and Presto are not one-trick ponies or just hive replacements. There's also the ability to query across multiple systems that is, to me, the feature that future proofs a lot of architectures. It inherently frees you up to fiddle with your data in different systems but keep the access to that system in one location.
Yup, I would say Athena is the most commonly used version of Trino/Presto out there.
Also, if anyone is reading this wondering which version Athena comes from, it's a release that was common to both Presto and Trino before the fork. However, more recently we've worked with the Athena team on getting the newer Trino features that aren't in Presto today in Athena. So it's starting to become more based on Trino.
It’s a query engine that can talk to multiple databases and run join queries across tables from multiple sources. It also runs as a faster alternative to Hive.
Starburst employee here so take my view for what it’s worth. A few points:
Starburst spun out from some contributors working on Teradata’s Presto distro in 2017.
The Presto creators left Facebook in late 2018 and worked on building the new community and fork for most of 2019. They joined Starburst around a year later.
I asked these same questions in my last company and it seems like nothing was correlated between starburst and the new fork. After talking with Martin T. (One of the creators of Presto) about it and a few other long time contributors and my concerns went away.
I ultimately started to contribute to Trino and over the years I’ve only seen a community dedicated to making a successful query engine. You can see that by the activity and growth of the project.
Feel free to remain a skeptic but the best way to know is joint the community and make your own assumptions. :)