Hi! Hammad here - Chroma’s CTO. First off, I have an immense amount of respect for the Turbopuffer team, they’ve build a solid product.
I understand your point. Chroma Cloud has been quietly live in production for a year, and we have been discussing this architecture publicly for almost two years now. You can see this talk I gave at the CMU databases group - https://youtu.be/E4ot5d79jdA?si=i64ouoyFMevEgm3U. Some details have changed since then. But the core ideas remain the same.
The business model similarities mostly fall out of our architecture being similar, which mostly falls out of our constraints with respect to the workload being the same. There are only so many ways you can deliver a usage based billing model that is fair, understandable, and predictable. We aimed for a billing model that was all three, and this is what we arrived at.
On aesthetics, that’s always been our aesthetic, I think a lot of developer tools are leaning into the nostalgia of the early PC boom during this AI boom (fun fact, all the icons on our homepage are done by hand!).
On differences, we support optimized regexes vs full-scans, lending better performance. We also support trigram based full-text search which can often be useful for scenarios which need substring matches. We also support forking, which allows for cheap copy-on-write clones of your data, great for dataset versioning and tracking git repos with minimal cost. We've been building with support for generic sparse vectors (in beta) which enables techniques like SPLADE to be used, rather than just BM25. You can also run Chroma locally, enabling low-latency local workflows. This is great for AI apps where you need to iterate on a dataset until it passes evals, and then push it up to the cloud.
Chroma is Apache 2.0 open source - https://github.com/chroma-core/chroma and has a massive developer community behind it. Customers can run embedded, single-node and distributed Chroma themselves. We've suffered from depending on closed-source database startups and wanted to give developers using Chroma confidence in the longevity of their choice.
Lastly, we are building with AI workloads front and center and this changes what you build, how you build it and who you build for in the long term. We think search is changing and that the primary consumer of the search API for AI applications is shifting from human engineers, to language models. We are building some exciting things in this direction, more on that soon.
Sure, I see. I think this is an area where complexity analysis doesn’t lead to useful information.
To be more correct it’s O(N/C log C) where C is the capacity of a segment. In this case you can ignore 1/C and log C as constant. So now sure, you actually just have O(N). But this is not super useful as it says that a segmented hnsw approach and brute force approach are the same - when this is really not the case in practice.
Also O(N log N) > O(N) so I’m not sure why we would ever do anything with segmentation according to that analysis if it were correct.
"The obstacle is that until now, off-the-shelf vector databases could not index a dataset larger than memory, because both the full-resolution vectors and the index (edge list) needed to be kept in memory during index construction. Larger datasets could be split into segments, but this means that at query time they need to search each segment separately, then combine the results, turning an O(log N) search per segment into O(N) overall."
I don't think its the role of the government to penalize layoffs. If a company needs to do a layoff to remain viable it must be allowed to - the downside (everyone loses their job) is worse. And trying to apply some mechanism where the government decides if its ok or not seems too intractable.
I think the approach of things like the WARN (https://en.wikipedia.org/wiki/Worker_Adjustment_and_Retraini...) act are more plausible. Use legislation to make the workforce more informed, resilient and ahead-of layoffs. You don't want mass layoffs crippling your economy because people are unable to plan.
It's a direct consequence of the WARN act that most tech layoffs have extremely generous severance.
Yeah, was thinking the same thing. Something deeper to be explored here esp due to the consistency across domains. I wonder how far we can push "externalizing" higher level information the model wants to store while in the forward pass.
Maybe I am misunderstanding but the postgres buffer cache is LRU and will evict pages right? At times data may go to disk and then during serving it will have to be loaded into memory? So it is quite dependent on the size of your buffer cache as well as contention for that buffer cache.
Also the cache access patterns will vary between these implementations and is worth considering.
No one is discrediting (or even discussing really) the validity of the point that if you use postgres pgvector or pgembedding may makes sense for the reasons you mention. The question was about how the algorithms compare, I compared the algorithms. Comparing ANN algorithms is relatively apples-to-apples, but is still complicated and filled with nuance.