Pgvector 0.4.0 Performance(supabase.com)
supabase.com
Pgvector 0.4.0 Performance
https://supabase.com/blog/pgvector-performance
4 comments
> What about offering Qdrant as an alternative to PG on the supabase platform
We are "all in" on Postgres, but we could definitely find ways to support qdrant (probably a Wrapper: https://supabase.com/blog/postgres-foreign-data-wrappers-rus...)
We are "all in" on Postgres, but we could definitely find ways to support qdrant (probably a Wrapper: https://supabase.com/blog/postgres-foreign-data-wrappers-rus...)
i am not Paul of course, but i just want to highlight that the idea behind blogpost was to show that if you set some settings for postgres correctly, you can achieve much better performance
ie you can notice that most of results are on the same hardware as in nirantk, and we achieved 7 times better RPS with 0.98 precision on it and 19 times better RPS with 0.91 precision on the same amount of cpu and memory
goal was to compare pgvector with pgvector rather than any other technology
ie you can notice that most of results are on the same hardware as in nirantk, and we achieved 7 times better RPS with 0.98 precision on it and 19 times better RPS with 0.91 precision on the same amount of cpu and memory
goal was to compare pgvector with pgvector rather than any other technology
hey hn, supabase ceo here
this is a response to a recent pgvector vs qdrant benchmark. As we mention in the post, we’re not trying to prove that pgvector is faster than Qdrant. Our goal is to show that pgvector/postgres scales predictably for the workloads that most companies need.
We scale it to ~1800 requests per second with 91% precision in this set of benchmarks. Still, it could be better. Luckily there are quite a few features and improvements planned for v0.5.0 [0]:
this is a response to a recent pgvector vs qdrant benchmark. As we mention in the post, we’re not trying to prove that pgvector is faster than Qdrant. Our goal is to show that pgvector/postgres scales predictably for the workloads that most companies need.
We scale it to ~1800 requests per second with 91% precision in this set of benchmarks. Still, it could be better. Luckily there are quite a few features and improvements planned for v0.5.0 [0]:
- Adding HNSW: an index with better speed & precision than IVFFlat (at a higher memory cost)
- Product quantization: better storage for IVFFLAT, improving speed and recall
- Parallel index builds: building your IVFFLAT indexes will be much faster
supabase engineer here,
would be happy to share details regarding our research and outcomes if you have any questions
would be happy to share details regarding our research and outcomes if you have any questions
Had a question about the "pre-warming" technique you used - why does this help? You say it helps with RAM utilization, but why?
Does it just spin up the right number of probes up so the experiments don't include this extra cost of spinning up these probes?
(apologies if this is a dumb question - I don't know enough about vector databases to know if this is a dumb question or not!)
Does it just spin up the right number of probes up so the experiments don't include this extra cost of spinning up these probes?
(apologies if this is a dumb question - I don't know enough about vector databases to know if this is a dumb question or not!)
[deleted]
What does "precision@10" mean in the graphs shown in the blog?
it means the precision at K(10) or precision for first 10 documents retrieved.
In other words - number of relevant documents amongst retrieved 10(k) of them
So 0.98 precision would mean that 9.8 out of 10 documents are relevant on average
So 0.98 precision would mean that 9.8 out of 10 documents are relevant on average
PS: What about offering Qdrant as an alternative to PG on the supabase platform? Just an idea. :)