Show HN: New Postgres Connection Pooling Extension Breaks 100k Barrier(medium.com)

11 pointsby jonahharris2 yıl önce3 comments

Show HN: New Postgres Connection Pooling Extension Breaks 100k Barrier

https://medium.com/@jonah.harris/surpassing-100k-postgres-connections-with-our-in-database-connection-pooling-extension-7084552234ea

If you’ve ever dealt with scaling PostgreSQL, you know the traditional process-per-connection model just doesn’t cut it when you're handling massive numbers of connections, particularly short-lived ones. When I led architecture at The Meet Group, which had one of the largest deployments of Postgres in the world, we optimized everything: connection pools in the app tier, as a man-in-the-middle, and even on the database server. It ran great, but it was a pain in the ass to manage and debug.

Now, as co-founder and CEO of NEXTGRES, I've been working on a better and more seamless solution: in-database connection pooling. Our new extension integrates directly into Postgres, removing the hassle of external setup. It’s designed to significantly enhance how Postgres manages high volumes of connections, making life a lot easier for anyone running large-scale instances.

While this is only our Alpha release, which reimagines Konstantin Knizhnik's old connection proxy patch as an extension, it surpasses 100k connections without significant degradation. Our second, not-yet-released Beta version incorporates a stripped-down embedded version of PgBouncer to handle several aspects better than the original patch. Both Alpha and Beta gave us insights into significant areas of improvement, which are being applied to our current production-grade version in development, soon to be released.

After talking to a few people who tried this old version out and thought it was a really cool extension, we decided to release it in the spirit of "Release Early, Release Often." Feel free to try it out, give us some feedback, and see what it's like to enjoy the benefits of connection pooling without the external configuration and management hassle.

That said, please avoid using this in a production environment as it was designed primarily for evaluation purposes, and we omitted aspects not relevant to our tests :)

2 comments

geenat2 yıl önce

Anything to cut down on the amount of moving parts required to operate postgres is a huge win in my books. Good job.

PeterZaitsev2 yıl önce

Is it Open Source ?

jonahharris2 yıl önce

Right now, the source is available under the Confluent license, but we intend to go BSD/MIT once our fundraising is done.