Very cool, gotta try this out! Curious, how does it handle blob data (e.g., images) -> Is there a way to display the image blobs in the widget too? Kind of like how Hugging Face displays Lance blobs in their dataset viewer: https://huggingface.co/datasets/lance-format/handwriting-ocr
Really cool stuff! Looking at the schema of the financial dataset, though, it's really flat, and it doesn't really leverage the power of the BAML schema format (which excels at representing nested data with fewer tokens than JSON). I suspect that the performance difference with and without the BAMLAdapter would have been more stark if the data model being extracted was more nested.
Great work though, and here's to more experiments of this nature with nested data extraction. Kudos to the BAML team for coming up with such an awesome format, and kudos to the DSPy maintainers for designing such a hackable, usable framework!
The semantic routing step with intent classification and query rewriting was fascinating. The fact that multiple indexes were used is also understated. Is there anywhere one can explore the internals?
To me they seem like the pythonic version of dbt! Instead of yaml, you write Python code. That, and a lot of on-the-fly computations to generate an optimized workflow plan.
Hi! IMO you should definitely check out Kuzu - It's an embeddable, fast, highly scalable graph database with an MIT (highly permissive) open source license. Kuzu was created for broader adoption in the open source community and scales to very large graphs on a single machine. Disclaimer: I work as DevRel at Kuzu. Check out our GitHub!
Kuzu is an open source, embedded graph database that supports the Cypher query language. We wrote a blog post that serves the dual role of explaining what MCP is and how the MCP server of a database system can be useful. Our demo is based on getting Cursor (or Claude Desktop) to debug a bug in a database (and not in the code itself, which LLMs can already do without any access to the database). The biggest advantage of this way of interacting with a local embedded database like Kuzu, is that a typical 4-5 step debugging session (where the developer would write manual Cypher queries to understand the data), can now be automatically done by an LLM with a single and very simple starter prompt.
Perhaps a lot of similar data engineering/analysis tasks (e.g., ETL, quantifying results, visualization) in the future can be much more easily delegated to LLMs with simple prompts via MCP servers of database systems (possibly multiple of them at the same time). It would be fascinating to see the programming patterns that emerge in this space as MCP clients/servers evolve alongside! We'd love to get feedback and hear more about how otler folks are getting LLMs to do cool stuff using database MCP servers!
Hi, glad to help! I'm a DevRel advocate at Kuzu, and have spent a decent amount of time in other database paradigms thinking about these things. I'm familiar with SurrealDB too.
Although I cannot comment too much SurrealDB's exact capabilities and performance at this point, I can definitely highlight that at the data modeling and query language-level: Kuzu's data model is a property graph model (so an actual "graph" model rather than a multi-model database) and Kuzu implements Cypher as its query language, which is already widely adopted in the industry and is very intuitive to write (for both humans and LLMs).
Although Surreal DB does indeed offer an embedded mode, Kuzu is by design 100% embedded, is super-lightweight and can run natively in many environments, such as browsers, Android applications, AWS Lambda (serverless) and we're especially designed to be a VERY Python-friendly graph database that integrates with pretty much all well-known Python libraries. Because of its columnar storage layer, Kuzu can seamlessly read and write different data formats, such as Panda/Polars DataFrame, Arrow tables, Iceberg or Delta Lake tables and seamlessly move data between advanced graph analytics libraries like NetworkX. For anything related to graph computation, Kuzu is likely to have all the right tools and utilities to help you solve the problem at hand.
In my opinion, it's a myth that databases are heavy, monolithic pieces of software, and hopefully, using Kuzu will demonstrate that it's totally possible to have data in your primary store but seamlessly move it to a performant graph storage layer when required, and move the results back with minimum friction and cost. Hope that helps!
Your concern makes sense, but in the demo we show, all your private data AND the graph database AND the LLM (basically, everything) is confined to your client session in the browser, and no data actually ever leaves your machine. That's the whole point of Wasm!
The graph that you build is more for your own exploration and not for sharing with the outside world.
Great point! Several years ago there was a project GRainDB, which along with GraphflowDB (a purely in-memory graph database) formed the ideas of what is now Kuzu :)
True, but there are likely innovations happening in multiple dimensions all at once: WebGPU improvements that better utilize a device's compute, Wasm64. And of course, LLMs over time become SLMs (smaller and smaller models), that can do a surprisingly large variety of things well.
Putting aside LLMs for a minute, even applications that do not need LLMs, but benefit from a graph database, can be unlocked to help build interactive UIs and visualizations that retain privacy on the client side without ever moving the data to a server. Loads of possibilities!
Hi, I work at Kuzu and can offer my thoughts on this.
You're making a fair observation here and it's true for any high level query language - SQL and Cypher and interchangeable unless the queries are recursive, in which case Cypher's graph syntax (e.g., the Kleene star * or shortest paths) has several advantages. One could make the argument that Cypher is easier for LLMs to generate because the joins are less verbose (you simply express the join as a query pattern). This post is not necessarily about graph analytics. It's about demonstrating that it's very simple to develop a relatively complex application using LLMs and a database fully in-browser, which can potentially open up new use cases. I'm sure many people will come up with other creative ways putting these fully in-browser technologies, both graph-specific, and not, e.g., using vector search-based retrieval. In fact, there are already some of our users doing this right now.
Very interesting! Could you elaborate a bit on the parallels with Keras, other than the API similarities? What aspects of working with LLMs does SynaLinks make easier?
Really exciting stuff! I can clearly see from the GitHub issue list that you do NOT intend to repeat the mistakes of LangChain, i.e. minimize bloat and abstraction level.
How would you say this compares to DSPy? At first glance, ell is very application developer focused, while DSPy is more designed as a lower level framework on top of which others can build. Curious to see how this evolves for ell. Also, prompt "optimization" as per the docs is a fuzzy term - what is being optimized exactly? Basically, if I want to minimize my time doing prompt engineering (which I hate), is ell the framework for me?
By this I presume you mean build a search index that can retrieve results based on keywords? I know certain databases use Lucene to build a keyword-based index on top of unstructured blobs of data. Another alternative is to use Tantivy (https://github.com/quickwit-oss/tantivy), a Rust version of Lucene, if building search indices via Java isn't your cup of tea :)
Both libraries offer multilingual support for keywords, I believe, so that's a benefit to vector search where multilingual embedding models are rather expensive.
Very good points. Have you seen any examples of systems (or projects) that successfully combine multiple backend data sources, including databases, that perform better than the single backend alone? This seems like an important enough question that it ought to have been documented somewhere.
Totally agree! The wave of blog posts and examples one sees where it's just text-to-SQL or text-to-Cypher or any other query lang aren't really exploring the topic at any level of technical depth, and we need to see more evaluations and technical papers that characterize them, so that we can understand how to build better systems.
This is an excellent article that asks some much-needed questions on the literature that exists connecting LLMs and RAGs on unstructured data, with knowledge graphs in between. We've seen plenty of articles that speculate on how one can build a simple retrieval system on top of a KG, but there are two challenges: a) constructing a high quality KG isn't easy, and b) keyword or phrase embedding on metadata for pre-filtering on relevant sections of the graph is required.
As some others here have pointed out, information extraction and searching with relevant context are the hardest parts of any search system, and it's clear that simply chunking vectors up and throwing them into a vector DB has limitations, no matter what the vector DB vendors tell you. Just like this article says, I hope that 2024 is the year where we actually get some papers that perform more rigorous evaluations of systems that use vector DBs, graph DBs, or a combination of them for building RAGs.
OpenNRE (https://github.com/thunlp/OpenNRE) is another good approach to neural relation extraction, though it's slightly dated. What would be particularly interesting is to combine models like OpenNRE or SpanMarker with entity-linking models to construct KG triples. And a solid, scalable graph database underneath would make for a great knowledge base that can be constructed from unstructured text.
Either way, thanks for sharing!