A couple of things:
1. There is a governance mode, so you can restrict the AI and force it to adhere to the established schema. If the AI tries to create a relationship with a type 'FRIEND_OF' one day, then another day it uses 'FRIENDS_WITH', it would block and force the AI to propose the new relationship type. This avoids polluting the schema with variations.
2. The schema could be manually added to a system prompt, but then you have to modify the system prompt everytime the schema changes. This system manages the schema. The AI knows the up to date version of it immediately.
One of the biggest issues I've had building with AI is giving it long-term memory that works across any knowledge domain and is simple enough for the agent to use without mistakes.
So I decided to solve this problem, and I'm putting it out there for anyone to use.
Why should you care?
If you are building agents, creating smart workflows, or need to store a lot of data for your AI, keep reading.
What is Deep Memory:
Deep Memory is an implementation of GraphRAG. If you are not familiar with Graph, it's a storage technology that uses vertices and edges (or Entities and Relationships if you like).
But, this one is a little different. I found that AI needs the proper guidance to work effectively with external systems. Heck that's why Anthropic created the MCP (Model Context Protocol) in the first place, to allow an AI to understand the tools.
So I extended this concept to the database itself. This one has a Vocabulary. It's kind of like giving a dictionary that explains the nodes and edges to the AI before it's done a single query. What this means is it knows all the entity types and relationships straight off the bat. In addition the system contains some instructions helping it to query the database efficiently. We call this progressive disclosure.
As you've probably encountered AI has a limited context window. We can't just give it everything. What we want to do, is help it get to just the context it needs quickly. Deep Memory does this by providing the Vocab when the AI opens a memory graph repository, and in usage instructions that helps it get started.
What can you do with Deep Memory?
1. You can connect it to your AI using the MCP tools available in NPM, such as Claude Desktop.
2. You can run it directly in a clone of the repository, and index your own content if you want to.
3. You can export/import your deep memory repositories as .dkg files, meaning they are very portable.
4. You can bypass the MCP tools completely and integrate the deep memory library directly in your own app or system. Anything that can support npm packages.
Here's some examples of what I've built with Deep Memory with outstanding results IMO:
1. I indexed council planning rules from PDF files and created an Expert AI Agent using a deep memory graph - it became an expert council planning advisor telling me exactly what permits I need, what rules apply.
2. A legal expert - I indexed a set of Legal Documents, building a Legal Memory graph. I then used this to help me refine, strengthen and improve legal documents. The AI was able to quickly see variations and very relevant context to aid the drafting of new clauses.
3. A mining equipment expert - By downloading the Caterpillar and Komatsu Equipment technical specs and manuals from the manufacturer websites, I created an expert in maintenance and equipment specs, compatibility for parts, procedures etc.
4. I fully implemented Deep Memory into our flagship AI Productivity platform at https://ucm.utaba.com.au. Along with a graph viewer, and the built in Agent system, I was able to get the agents up and running in no time flat! (quick plug, reach out if you want to work with me)
Looking for feedback on this. What domain would you point this at? The council, legal, and mining ones surprised me with how well they worked — I'm curious where else this lands.
Nice write up, thanks. When I used claude to do some pen testing for one of my apps it initially refused. After I explained and demonstrated I'm the author, it reasoned through it and allowed it.
I've been a software developer for 30 years. I've loved getting into AI and working with it to code, but here is the truth:
I plan the solution architecture, orchestrate the AI to code, verify the code it creates, then point out all the security, scalability and design flaws it introduced.
So can I be replaced by an AI? Not yet at least, but, I am made more efficient.
Does that equate to job loses? Possibly. A development team of 10 people, once patterns and practices are created, become more efficient. They either produce more, or some team members might be let go and output is maintained.
2. The schema could be manually added to a system prompt, but then you have to modify the system prompt everytime the schema changes. This system manages the schema. The AI knows the up to date version of it immediately.