Prequel enables software companies to sync data to their customers' data environments, at massive scale. With the rise of agents, syncing data to customers' data environments is becoming table-stakes for a lot of software companies. We make that incredibly easy for them.
We're a team of four engineers based in NYC. We're cash-flow positive and growing fast. We're solving a number of hard technical problems that come with syncing hundreds of billions of rows of data every day with perfect data integrity: building reliable & scalable infrastructure, making data pipelines manageable without domain expertise, and creating a UX that abstracts out the underlying complexity to let the user share or receive data. We're powering this feature at companies like Stripe (Metronome), Gong, Iterable, and more.
Our stack is primarily Golang/K8s/Postgres/DuckDB/React/Typescript and we support deployments in both our public cloud as well as our customers' clouds. Due to the nature of the product, we work with nearly every data warehouse product and most of the popular RDBMSs.
Agreed, I like `ingest` as well. It does somewhat violate principle d), but the other solutions violate more of those. And to your point, they're principles, not rules.
Prequel | Staff Frontend / Full Stack Engineer / SWE Intern | ONSITE in New York City | Full Time | https://prequel.co
- Prequel is the customer data access platform. We enable SaaS companies like LaunchDarkly, Gong, and LogRocket to make data accessible to their customers.
- Transferring trillions of rows between data stores every month.
- Revenue is up 50% since Jan 1.
- We're launching a new product which is quite frontend heavy, and want to bring a pro onboard who can act as tech lead for it. This is not your standard CRUD app -- a big part of it is building complex SDKs and APIs used by engineers at top-tier companies.
- Frontend in Typescript/React, backend in Go.
- Team is stacked, with alums from Stripe, GIPHY, Google, Flatiron Health, and more.
Disagree on the data silo issue. There's a growing trend of SaaS providers making data available to their customers by feeding it back into their DWs. It started with the likes of Segment and Heap, and has now grown to include companies like Stripe, Salesforce, and Zuora to name a few. I'd wager that making data accessible is only going to become more table-stakes over time.
That's a bit of a strawman argument. Per the post, you can't leverage this on a read replica, it has to be run on primary. So you're going to stand up and manage a full new Postgres instance for this?
I'm sure there are many cases when that makes sense, but there are many cases when that's also overkill. An in-memory cache inside your server will give you better performance, and a lot of less infrastructure maintenance complexity.
Why wouldn't you simply use SQLite (or some other in-memory flavor of SQL) instead of hacking the main Postgres db and adding load to the primary instance?
The author makes a valid point that there's something nice about using familiar tooling (including the SQL interface) for a cache, but it feels like there are better solutions.
Prequel | https://prequel.co | Senior/Staff Software Engineer | Full Time | GoLang, Postgres, Typescript, React, K8s | $150k-$200k + equity | ONSITE in NYC
Prequel is an API that makes it easy for B2B companies to sync data directly to their customer's data warehouse, on an ongoing basis.
We're solving a number of hard technical problems that come with syncing tens of billions of rows of data every day with perfect data integrity: building reliable & scalable infrastructure, making data pipelines manageable without domain expertise, and creating a UX that abstracts out the underlying complexity to let the user share or receive data. We're powering this feature at companies like LogRocket, Modern Treasury, Postscript, and Metronome.
I don't thing there's necessarily anything there. Microsoft might be burning money because they've decided that browser adoption and usage is worth it to them. It doesn't have to involve OpenAI in any way.
Prequel | https://prequel.co | Senior Software Engineer | Full Time | GoLang, Postgres, Typescript, React, K8s | $150k-$180k + equity | ONSITE in NYC
Prequel is an API that makes it easy for B2B companies to sync data directly to their customer's data warehouse, on an ongoing basis.
We're a tiny team of four engineers based in NYC. We're solving a number of hard technical problems that come with syncing tens of billions of rows of data every day with perfect data integrity: building reliable & scalable infrastructure, making data pipelines manageable without domain expertise, and creating a UX that abstracts out the underlying complexity to let the user share or receive data. We're powering this feature at companies like LogRocket, Modern Treasury, Postscript, and Metronome.
Our stack is primarily K8s/Postgres/DuckDB/Golang/React/Typsecript and we support deployments in both our public cloud as well as our customers' clouds. Due to the nature of the product, we work with nearly every data warehouse product and most of the popular RDBMSs.
We're looking for a full stack engineer who can run the gambit from CI to UI. If you are interested in scaling infrastructure, distributed systems, developer tools, or relational databases, we have a lot of greenfield projects in these domains. We want someone who can humbly, but effectively, help us keep pushing our level of engineering excellence. We're open to those who don't already know our stack, but have the talent and drive to learn.
The "how do we make money" section on their website is interesting.
> We have not made any decisions about how we may charge for the product in the future. That said, we believe your personal AI should always be directly aligned to your interests. We therefore think it's crucial that you are the only person who pays for it, so that will likely be our primary default business model. However, it’s still early days for this new technology. We also recognize that some people would rather access a free service and would prefer to see adverts in return.
I'm sympathetic to the idea that startups need to iterate on their business model to be successful. At the same time, this sounds a whole lot like "we promise that our business model doesn't rely on selling your data, unless we decide otherwise."
We can detect deleted rows for incremental transfers (and propagate those) if they're soft-deleted in the source, whether through a deleted_at column or a is_deleted column.
For now, we only support maintaining current state in the target.
Yup! We support all common cloud file storage as destinations (S3, R2, GCS, Azure Blob Storage) as well as vanilla SFTP servers.
That can be part of the value-add, though for on-prem deployments, we never touch the credentials ourselves.
Not to sound like a consultant, but there's three value-adds I'd call out:
1. Handling the dialect, types, and connection modalities of many different databases. This takes a lot of time to build and there's a lot of nuance that's non-trivial to work through.
2. Replicating data and guaranteeing data integrity + reliability. There's again a lot of nuance here, especially once you start considering that data is eventually consistent in most sources, that you want to transfer it as efficiently as possible, etc.
3. Providing a clean UX that end-customers can use out of the box, such that the end-customer experience is clean and intuitive. We spend a lot of time thinking about how it makes sense for people to connect their data, so that our customers don't have to.
It depends -- mostly on whether the vendor (the company receiving the data) is comfortable requiring the source to map some fields.
For low volume cases, we can operate with zero mapping of fields. In those cases, we run every transfer as a full refresh.
If the volumes are higher, then we'll typically ask the source to expose a primary key and last_updated_at timestamp field. In those cases, we run incremental transfers. We use the last_updated_at to figure out what data to transfer, and the primary key to merge it into the destination table without creating dupes.
Not sure if I'm understanding the analogy. The way I usually describe it is that it's like Census / Hightouch, but it's offered by the vendor as a first-party feature.
Let's take Salesforce as an example. Let's say they want to pull in data from their customer's database -- maybe so that sales reps can keep track of how much volume the customer did in the last month -- instead of requiring the customer to instrument their code with Salesforce API calls. Salesforce could use this tool to connect directly to all of their customer's databases / data warehouses, regardless of whether they're Postgres, Snowflake, Clickhouse, etc.
As far as why it's non-trivial: you have to support a lot of different databases / data warehouses, which all have slightly different query languages, type systems, and optimizations. Then you've got to move the data reliably, dealing with things like eventual consistency etc. We feel like that's the reason this hasn't been built yet.
We're big fans of DuckDb at https://prequel.co! We use it as part of our own dataframe implementation in Go. The speed is unbeatable and the tool is top notch. There are a few rough edges (it's not quite 1.0 level of stability yet), but the team is super reactive and has fixed bugs we've reported in < 48hrs pretty much every time.
No near-term plan for Salesforce support from us, but that's an interesting use-case. Definitely check out the other tools folks mentioned if you haven't already, they should be able to get you most of the way there.
Prequel enables software companies to sync data to their customers' data environments, at massive scale. With the rise of agents, syncing data to customers' data environments is becoming table-stakes for a lot of software companies. We make that incredibly easy for them.
We're a team of four engineers based in NYC. We're cash-flow positive and growing fast. We're solving a number of hard technical problems that come with syncing hundreds of billions of rows of data every day with perfect data integrity: building reliable & scalable infrastructure, making data pipelines manageable without domain expertise, and creating a UX that abstracts out the underlying complexity to let the user share or receive data. We're powering this feature at companies like Stripe (Metronome), Gong, Iterable, and more.
Our stack is primarily Golang/K8s/Postgres/DuckDB/React/Typescript and we support deployments in both our public cloud as well as our customers' clouds. Due to the nature of the product, we work with nearly every data warehouse product and most of the popular RDBMSs.
Apply here: https://www.ycombinator.com/companies/prequel/jobs/VNoKffl-s... or email jobs (at) prequel.co and reference this post.