hey, supabase engineer here; we didn’t check that out with files that small, but thanks for the idea, i will try it out
the only thing i can say related to the topic is that s3 multipart outperforms other methods for files larger than 50mb significantly, but tends to have similar or slightly slower speeds compared to s3 regular upload via supabase or simplest supabase storage upload for files with size about and less than 50mb.
s3-multipart is indeed the fastest way to upload file to supabase with speeds up to 100mb/s(115 even) for files>500mb. But for files about 5mb or less you are not going to need to change anything in your upload logic just for performance cause you won’t notice any difference probably
interesting quote: "A vital component of Foundations is its robust and ergonomic API for seccomp, a Linux kernel feature for syscall sandboxing. This feature enables the setting up of hooks for syscalls used by an application, allowing actions like blocking or logging. Seccomp acts as a formidable line of defense, offering an additional layer of security against threats like arbitrary code execution."
hey hn, supabase engineer and blogpost coauthor here.
we made our first experiments with HNSW index for vector search and noticed about 5 times better performance compared to IVF with high dimensional vectors, such as embedding-ada by OpenAI.
we haven’t included results for smaller models like gte-small (384d) just yet, but we’re currently running those benchmarks as i write this comment. in our smoke tests the difference in performance isn’t as pronounced, but it still appears promising, suggesting that switching to HNSW could be beneficial for the majority of use cases.
oh there have been improvements with IVF as well, with index building times decreasing by roughly half. So, if you’re considering sticking with IVF, it’s advisable to upgrade to the latest version for these enhancements.
you can find extended version of ann testing framework in the GitHub [0]. its just the original one with a lil bit of code for pgvector and vecs lib.
even networking alone adds about 1ms or so, and these are 2 hops each side: lb - sv - db, instead of the same instance for pgb scenario
another 1ms goes for a gain of a millions connections, multitenancy, and a lot of features that are possible to implement and are planned to implement.
i am curious what usecase do you have, that 2ms is unacceptable? in most cases it would be insignificant to network latency to the end client
it means the precision at K(10) or precision for first 10 documents retrieved.
In other words - number of relevant documents amongst retrieved 10(k) of them
So 0.98 precision would mean that 9.8 out of 10 documents are relevant on average
i am not Paul of course, but i just want to highlight that the idea behind blogpost was to show that if you set some settings for postgres correctly, you can achieve much better performance
ie you can notice that most of results are on the same hardware as in nirantk, and we achieved 7 times better RPS with 0.98 precision on it and 19 times better RPS with 0.91 precision on the same amount of cpu and memory
goal was to compare pgvector with pgvector rather than any other technology
is it the right convo branch?
as this is a chatgpt plugin and uses openai embeddings api, it for certain sends some data to their servers.
both when embeddings are generated and when chatgpt sends request to your plugin and receives response back.
Hey, co-author here, thanks for caring about me. This is me who added myself as an author to the blog, i feel like “engineering” is the best description for the work I do in Supabase :)
You can find it in PR https://github.com/supabase/supabase/pull/14520
We have a flat culture and if you scroll through past posts, you see that it is the most common description for our team.
As for the work behind, copple made significant edits and additions (as I thought of them) to my draft, so i decided add him as coauthor (just to make sure: I wasn’t asked by anyone and copple only noticed this after publication)
I just added a PR to the great retrieval plugin template repo from OpenAI to add Datastore options for Supabase and Postgres. This is an implementation of Datastore interface using Postgres database and 2 different interfaces: PostgREST for using with the ability to hook everything with your existing app and RLS policies, or just a plain database connection. They are completely swappable, the only thing you need to change is a couple of environment variables to switch from Postgres to Supabase and vice versa.
You will find an example of a plugin with full PostgresDB documentation embedded into postgres database in the post. And i will be happy to answer your questions here :)
Hey, from what we’ve seen up to 50k per node should be ok for supavisor on 4 cores as well. And TPS is here mainly to demonstrate that we not only create dummy connections but that these connections are really used. And 20k TPS is just the maximum amount that database node was able to handle. To bump tps we had to spin up more db replicas. We want to run these tests as well at some point, but now were just verifying that we can connect millions of clients and with some tangible tps but not overcomplicating the setup too much.
Hi, i was running these million connections benchmarks, happy to answer if you have any questions regarding this:)
In short, that was pretty easy to do, only things we had to tweak to make it work were some elixir and linux limits
I like C4 for system design overview and sequence diagrams to show detailed process of interactions in one particular scenario.
I found these two the most useful for me.
I also like to write diagrams as code with https://plantuml.com/sequence-diagram cause you can then add it to git and easily version it. Also, a lot of tools on the market have support for PlantUML