Edge-compatible Serverless Driver for Postgres(neon.tech)
neon.tech
Edge-compatible Serverless Driver for Postgres
https://neon.tech/blog/serverless-driver-for-postgres/
5 comments
Right: at a basic level, using WebSockets lets us change as little as possible from the user perspective. You get an ordinary Postgres session with an ordinary Postgres driver, full control over transactions, and so on.
At this point, your serverless function establishes a new Postgres connection on each call. We do pooling on the server side with pgBouncer, which means we can handle lots of simultaneous connections (which is what this approach generates). It's true that this approach doesn't fully optimise for low latencies. But, as Nikita has mentioned elsewhere in this thread, we have a roadmap for bringing latency down in a number of different ways over time.
At this point, your serverless function establishes a new Postgres connection on each call. We do pooling on the server side with pgBouncer, which means we can handle lots of simultaneous connections (which is what this approach generates). It's true that this approach doesn't fully optimise for low latencies. But, as Nikita has mentioned elsewhere in this thread, we have a roadmap for bringing latency down in a number of different ways over time.
Ok, so to make that explicit: If I want to do 5 parallel queries on my serverless function I should still have a connection pool size of 5 in my application, which will be fine as PgBouncer ensures there are plenty connections to open and use from the database server side. Correct?
In principle, yes, but as things stand you'll be starting 5 separate TLS connections to the server that way. My feeling is that this would be an unusual way to use serverless functions. Is it something you think you'd do?
Yes. You only have to open these connections once on the first execution of that function (cold start), any future request that hit this warm function will have 5 open connections and can instantly execute these queries in parallel. No overhead at all to open the connections.
What would be the alternative? Only execute the queries in sequence, one by one?
What would be the alternative? Only execute the queries in sequence, one by one?
HTTP does not (easily) allow for using answers of earlier queries in the transaction. E.g.
BEGIN;
INSERT INTO my_table (...) RETURNING (id);
SELECT count(\*) AS my_count FROM my_table;
INSERT INTO historical (new_id, value_derived_from_count, now());
COMMIT;
That I understand.
But of course for the user of the driver it might be fine if that is 2 or more HTTP requests. I expect that is how PlanetScale does it in their transaction implementation, https://github.com/planetscale/database-js#transactions, and I know that is how Prisma Data Proxy handles it - the transaction is identified with an ID which is returned to the Client and then included in further requests for the same transaction.
It's valid tradeoff to make to prefer a persistent connection to keep the overhead for multiple queries in a transaction as low as possible - which seems what Neon has done here.
But of course for the user of the driver it might be fine if that is 2 or more HTTP requests. I expect that is how PlanetScale does it in their transaction implementation, https://github.com/planetscale/database-js#transactions, and I know that is how Prisma Data Proxy handles it - the transaction is identified with an ID which is returned to the Client and then included in further requests for the same transaction.
It's valid tradeoff to make to prefer a persistent connection to keep the overhead for multiple queries in a transaction as low as possible - which seems what Neon has done here.
> the transaction is identified with an ID which is returned to the Client and then included in further requests for the same transaction.
Yes, and there's a catch there that people might not notice: HTTP-based queries utilize this ID to identify which active transaction to use, but this is vulnerable to concurrent HTTP requests on the same transaction ID, thus allowing query injection (early COMMIT, SELECT sleep(1000000), etc. by attackers) due to requests being re-routed every time you send the query.
Keeping a direct connection (albeit proxied) to the PostgreSQL instance prevents this kind of attack, while also allowing for better state keeping control in systems that don't have access to raw TCP sockets, and reducing per-query overheads.
Yes, and there's a catch there that people might not notice: HTTP-based queries utilize this ID to identify which active transaction to use, but this is vulnerable to concurrent HTTP requests on the same transaction ID, thus allowing query injection (early COMMIT, SELECT sleep(1000000), etc. by attackers) due to requests being re-routed every time you send the query.
Keeping a direct connection (albeit proxied) to the PostgreSQL instance prevents this kind of attack, while also allowing for better state keeping control in systems that don't have access to raw TCP sockets, and reducing per-query overheads.
In our case, it's a bit more state than just a transaction ID, but the same concept applies.
And yeah, to mirror the other comment, this isn't particularly something I'd consider a security issue. If an attacker has access to your transaction (session) state, this is a similar risk vector on any web service that utilizes browser cookies or sessions. If someone hijacks your browser cookies, they can do anything they want into your account typically, ignoring 2FA and extra layers.
Same risk here. If someone were to intercept your request, which is strictly over HTTPS, and hijack your session, or you erroneously were logging the session and authentication credentials publicly, then you're exposing yourself to a lot bigger risk than someone adding a sleep() into your session. They could just dump everything from the database, because they are you.
A direct connection only changes this because it's a stateful protocol, and if the connection is severed, everything is lost/rolled back and nobody can intercept that.
But anyways, I'd hardly call a well established security model of HTTP sessions flawed in this regard.
If anything, it's possible for a client to do some dumb things, like, parallel queries within a single transaction, which you cannot do with a direct connection, which ultimately is just... going to be undefined behavior since they fundamentally get serialized into the database, just in an undeterministic order relative to the client.
And yeah, to mirror the other comment, this isn't particularly something I'd consider a security issue. If an attacker has access to your transaction (session) state, this is a similar risk vector on any web service that utilizes browser cookies or sessions. If someone hijacks your browser cookies, they can do anything they want into your account typically, ignoring 2FA and extra layers.
Same risk here. If someone were to intercept your request, which is strictly over HTTPS, and hijack your session, or you erroneously were logging the session and authentication credentials publicly, then you're exposing yourself to a lot bigger risk than someone adding a sleep() into your session. They could just dump everything from the database, because they are you.
A direct connection only changes this because it's a stateful protocol, and if the connection is severed, everything is lost/rolled back and nobody can intercept that.
But anyways, I'd hardly call a well established security model of HTTP sessions flawed in this regard.
If anything, it's possible for a client to do some dumb things, like, parallel queries within a single transaction, which you cannot do with a direct connection, which ultimately is just... going to be undefined behavior since they fundamentally get serialized into the database, just in an undeterministic order relative to the client.
I am not sure I am following.
We can have either parallel queries in a transaction where the order does not matter, or we can have them serialized - but then the next query will only be sent onto its journey after the previous query returned its data to the app.
What are the attack vectors here? And where would the attacker sit and try to achieve what exactly?
We can have either parallel queries in a transaction where the order does not matter, or we can have them serialized - but then the next query will only be sent onto its journey after the previous query returned its data to the app.
What are the attack vectors here? And where would the attacker sit and try to achieve what exactly?
[deleted]
It’s not a trade off. They’ve not been able to do something that is doable and are trying to tell you it is a trade off.
Just turn it into a single query using CTEs.
Another angle here is compatibility. With our current driver one can use ordinary node-postgres package, as we can substitute TCP-related calls with WebSocket calls during the build time. With that it possible to use all the packages that do require node-postgres like Prisma, Zapatos, etc.
Prisma does not actually use node-postgres, but a Rust PostgreSQL driver. Prisma will not be able to use the Neon serverless driver.
That is only used in tests :) The query engine uses this: https://github.com/prisma/quaint/blob/6532d69b5aec007ad06ac6...
(I work at Prisma, could have mentioned that earlier)
(I work at Prisma, could have mentioned that earlier)
Gotcha. I drew my conclusion based on the mentioned package.json. Now wonder why did you decide to go with rust for query engine? Do you compile it into wasm?
It made sense at the time. We do not only support Node, but also have community Clients in Go, Python or Rust. Right now we are moving more and more parts from a Node-API library or binary engine (the two variants we support until now) over to Wasm modules where it is possible for our Node/TS/JS Client. Socket/TCP connections itself are unfortunately not supported yet, so this will only be partial. And maybe there is also a future where we support Node based databases drives. As this blog post we are commenting on shows, sometimes we have to combine the weirdest things together to achieve our goal.
While you're here, I want to mention that I never understood why Prisma couples so much the query builder (which IMO is the really best part of Prisma) to the query engine / DB driver (which is not differentiated at all as a product). I have 1000 ideas of really insanely cool thing we could do if Prisma could just spit up SQL query strings, or at least connect to a custom driver.
Prisma often does multiple queries when getting some specific data, the first only leading to the second and so on. But we might very well get there one day. You are not the only one asking for this: https://github.com/prisma/prisma/issues/5052
I’m Neon CEO. Happy to answer questions. We also have an interesting roadmap for the driver where we hope to keep driving latency and number of round trips down
Would be really cool to see you guys become a managed service on Fly.io - see what they did with Upstash & Redis:
https://community.fly.io/t/preview-managed-upstash-redis-wit...
They don't have a managed postgres offering yet ("this is not a managed postgres" - https://fly.io/docs/postgres/getting-started/what-you-should...)
https://community.fly.io/t/preview-managed-upstash-redis-wit...
They don't have a managed postgres offering yet ("this is not a managed postgres" - https://fly.io/docs/postgres/getting-started/what-you-should...)
I’m curious why not implement websockets into Postgres natively instead of relying on a proxy server? Wouldn’t that be the way to minimize latency?
Being able to connect to arbitrary Postgres servers (or potentially even the postgres compatibility-layers for AWS's Aurora or Google's AlloyDB) seems like a good reason to go for the proxy approach.
Yes. Also it was an easier thing to do and ship quickly. We are going to get very tight on latencies and if that would require building it into Postgres we will do so
Does Neon scale down storage? I'm paying GCP a lot for storage I don't use, and it's incredibly frustrating - they scaled me up no problem but won't let me scale down.
Storage is multi tenant and bottomless. Unused storage spills to S3 and is cheaper. We pass savings to our users
Hey Nikita, what are the key differences between neon and Supabase?
I think we innovate on different parts of the stack. Neon is offering separation of storage and compute and serverless. Theoretically you can run Supabase on top of Neon
To elaborate Neon is designed deliberately to plug into the Postgres ecosystem. So all of the ecosystem works out of the box. Supabase seems to be vertically integrated. But Paul should comment on that
I believe most of the tools that Supabase use would work with Neon, with the exception of Realtime. As I understand it, Neon doesn't support Logical Decoding yet[0], which is required so that the Realtime engine can receive updates from the database.
[0] https://community.neon.tech/t/plans-for-logical-replication/...
[0] https://community.neon.tech/t/plans-for-logical-replication/...
(Another one would be the GraphQL extension: https://github.com/supabase/pg_graphql/)
You are right. We will make it work soon
Just for the recerence, it is possible to use Cloudflare Tunnel to connect to Postgres from Cloudflare Workers [1]. IIRC they also use WebSockets for communication between the Worker and the Tunnel.
[1] https://developers.cloudflare.com/workers/tutorials/query-po...
[1] https://developers.cloudflare.com/workers/tutorials/query-po...
Hmm. Wouldn't this also be useful for browser-hosted application code that wants to talk to PG directly?
Right, that is one of the use cases. Auth is tricky part in that setup -- in the browser user may change the query in arbitrary way. But there are some workarounds like obtaining JWT from 3rd party service and checking it via Row-Level Security policy in Postgres.
Would it ever be possible to run neon on a mobile phone, offline?
We have some ideas of compiling Postgres to wasm and sync with the mothership in the cloud. But we haven’t funded the work. Check out electricsql they are looking at this problem.
Hey, link for the curious: https://electric-sql.com
Hybrid serverless <> local-first is the future. Neon in the client would be so cool.
Hybrid serverless <> local-first is the future. Neon in the client would be so cool.
Other serverless drivers or database APIs (PlanetScale Serverless Driver, AWS RDS Proxy or Prisma Data Proxy) usually use HTTP to replace the stateful TCP connection. The post explains a bit why this might be beneficial (can use existing tools, and use connection state) - but what about the other benefits and use cases of going with HTTP?
For example in that architecture the connection pool also moves from the application to the component in between and all that complexity can completely be removed from the app. Especially on serverless platforms, where in theory each request gets its own application with its own pool, that is huge. How does that work here? Or does the pool also move to the "WebSocket-to-TCP proxy" and you just did not highlight it in the post? (The graphic shows a "Connection Pool" but I can not really make sense of what that means.)