There are significant differences in performance, accuracy, and other aspects when using a general-purpose database like PostgreSQL. These differences can lead to bottlenecks in performance and data scale. You can refer to this comparison article to learn more
https://myscale.com/blog/myscale-vs-postgres-opensearch/
The specialized vector database performs well when processing pure vector tasks but performs badly when it comes to SQL compatibility and integration with the existing system; And the traditional database with vector algorithm or vector plug-in like ES, PG, and Redis, achieves the vector function, the advantage is that it is very easy to create tasks in a production environment, but when the data scale is relatively large, they will quickly encounter performance bottlenecks.
There is a new type of vector database that combines the best of both worlds, which is MyScale, the SQL vector database. You can refer to the following blogs to see the comparison. our comprehensive benchmark evaluation reveals that MyScale exceeds other products in terms of filtered vector search accuracy, performance, cost-efficiency, and index build time by a long way. Importantly, MyScale is the only product tested that delivers healthy search accuracy and QPS across various filter ratios.
Hey coummunity, we are thrilled to introduce you an open-source advanced SQL vector database — MyScaleDB!
With AI thriving, diverse data can now be represented uniformly using vectors and graphs for tasks like similarity retrieval, supporting various AI-generated business functions.
Existing vector databases come in two main types: pure vector databases like Pinecone and Qdrant excel in vector tasks but struggle with complex SQL queries and compatibility with mature databases. On the other hand, traditional databases with vector algorithms or plugins like Elasticsearch (ES), PostgreSQL (PG), and Redis offer simplicity but face performance issues with large datasets.
To bridge this gap, we're developing MyScaleDB, an AI database that combines the strengths of both approaches.
Some of the most significant benefits of using MyScaleDB include:
* Fully SQL-Compatible
- Fast, powerful, and efficient vector search, filtered search, and SQL-vector join queries.
- Use SQL with vector-related functions to interact with MyScaleDB. No need to learn complex new tools or frameworks – stick with what you know and love.
* Production-Ready for AI applications
- A unified and time-tested platform to manage and process structured data, text, vector, JSON, geospatial, time-series data, and more.
- Improved RAG accuracy by combining vectors with rich metadata and performing high-precision, high-efficiency filtered search at any ratio1.
* Unmatched performance and scalability
- MyScaleDB leverages cutting-edge OLAP database architecture and advanced vector algorithms for lightning-fast vector operations.
- Scale your applications effortlessly and cost-effectively as your data grows.
To quickly get started with MyScaleDB, simply pull and run MyScaleDB Docker Image:
docker run —name myscaledb myscale/myscaledb:1.4
Then run the following to enter the SQL shell:
docker exec -it myscaledb clickhouse-client
We can't wait to see more impressive AI applications built with MyScaleDB!
MSTG combines the best parts of both hierarchical graph and tree structures in its design.
A graph algorithm excels at initial convergence, and is usually faster at unfiltered search. However, its efficiency is severely hampered in filtered search. On the other hand, tree algorithms are slower and have lower accuracy for unfiltered search, but tree traversal is unaffected by filtered elements and retains performance for filtered search, as illustrated in the figure below. Therefore, combining these two algorithms in MSTG yields high performance and high accuracy for both cases and achieves fast index build time.
ShardingSphere-Agent provides the tracing plugin that can obtain tracing information of SQL parsing and execution using visualization tools Zipkin and Jaeger.
I will go back to my office next week from WFH status. After almost 4 months WFH, I feel excited to work at the office because it's really hard to separate my work from life which made me more tired, although to be honest, sometimes I feel freer to work from home. And when you work from home , it seems like you are always online because the time you get off work is not so clear to your colleagues unless you strictly set your time.
I'm a beginner for this field, so I have some questions. Is it spontaneous for the machine/server to pull data from the queue, run reports, and so on to perform batch operations? What is the extracted data used for? Isn't this action responding to a request?
I really agree with your opinion, especially the words in the last paragraph. I am working in a startup now for half a year, in which all the rules you mentioned is done well, so as a fresh graduate, I never regret working for a startup. There, I have learned a lot of different skills that definitely will be my great treasures in the future. So, for the fresh graduates, I think working for a startup makes great sense that helps them find their true value.
The condescending language indeed makes people uncomfortable, especially for the newbie of this field. Funny experience and a good guide that encourages me in doing the same thing as the author: remove the condescending language from various documentation.