Show HN: I replaced vector databases with Git for AI memory (PoC)

[alexmrv]

Hey HN! I built a proof-of-concept for AI memory using Git instead of vector databases.

The insight: Git already solved versioned document management. Why are we building complex vector stores when we could just use markdown files with Git's built-in diff/blame/history?

How it works:

Memories stored as markdown files in a Git repo Each conversation = one commit git diff shows how understanding evolves over time BM25 for search (no embeddings needed) LLMs generate search queries from conversation context Example: Ask "how has my project evolved?" and it uses git diff to show actual changes in understanding, not just similarity scores.

This is very much a PoC - rough edges everywhere, not production ready. But it's been working surprisingly well for personal use. The entire index for a year of conversations fits in ~100MB RAM with sub-second retrieval.

The cool part: You can git checkout to any point in time and see exactly what the AI knew then. Perfect reproducibility, human-readable storage, and you can manually edit memories if needed.

GitHub: https://github.com/Growth-Kinetics/DiffMem

Stack: Python, GitPython, rank-bm25, OpenRouter for LLM orchestration. MIT licensed.

Would love feedback on the approach. Is this crazy or clever? What am I missing that will bite me later?

[BenoitP]

I'm failing to grasp how it solves/replaces what vector db were created for in the first place (high-dimensional neighborhood searching, where the space to be searched grows by distance^dimension)

[aszen]

It doesn't replace vector db, it's more for storing agentic memory, think of information which you would like agents to remember across conversations with users just like humans

[alexmrv]

Super simplistic example, but say i mention my Daughter, who is 9.

Then mention she is 10,

a few years later she is 12 but now i call her by her name.

I have struggled to get any of the RAG approaches to handle this effectively. It is also 3 entries, but 2 of them are no longer useful, they are nothing but noise in the system.

[petesergeant]

Exactly. If you're not using embeddings, why would you need a vector db?

> Why are we building complex vector stores

Because we want to use embeddings.

[OutOfHere]

The submission and the readme fail to explain the important thing, which is how BM25 is run. If it creates bags of words for every document for every query, that would be ineffective. If it reuses the BM25 index, it is not clear when it is constricted, updated, and how it is stored.

Because BM25 ostensibly relies on word matching, there is no way it will extend to concept matching.

[cchance]

Somehow i feel like the op doesn't get what vector DB's are actually for, semantic search isn't the same as a markdown memory file

[lsb]

Interesting! Text files in git can work for small sizes, like your 100MB.

That is what's known in FAISS as a "flat" index, just one thing after another. And obviously you can query by primary key to the key-value store that is git, and do atomic updates as you'd expect. In SQL land this is an unindexed column, you can do primary key lookups on the table, or you can look through every row in order to find what you want.

If you don't need fast query times, this could work great! You could also use SQL (maybe an AWS Aurora Postgres/MySQL table?) and stuff the fact and its embedding into a table, and get declarative relational queries (find me the closest 10 statements users A-J have made to embedding [0.1, 0.2, -0.1, ...] within the past day). Lots of SQL databases are getting embedding search (Postgres, sqlite, and more) so that will allow your embedding search to happen in a few milliseconds instead of a few seconds.

It could be worth sketching out how to use SQLite for your application, instead of using files on disk: SQLite was designed to be a better alternative to opening a file (what happens if power goes out while you are writing a file? what happens if you want to update two people's records, and not get caught mid-update by another web app process?) and is very well supported by many language ecosystems.

Then, to take full advantage of vector embedding engines: what happens if my embedding is 1024 dimensions and each one is a 32 bit floating point value? Do I need to save all of that precision? Is 16-bit okay? 8-bit floats? What about reducing the dimensionality? Is it good enough accuracy and recall if I represent each dimension with an index to a palette of the best 256 floats for that dimension? What about representing each pair of dimensions with an index to a palette of the best 256 pairs of floats for those two dimensions? What about, instead of looking through every embedding one by one, we know that people talk about one of three different topics, and we have three different indices for each of those major topics, and to find your nearest neighbors you want to first find your closest topic (or maybe closest two topics?) and then search in those lower indices? Each of these hypotheticals is literally a different “index string” in an embedding search called FAISS, and could easily be thousands of lines of code if you did it yourself.

It’s definitely a good learning experience to implement your own embedding database atop git! Especially if you run it in production! 100MB is small enough that anything reasonable is going to be fast.

[mingtianzhang]

Very interesting idea! I also replaced vector databases with "Table of Content" for retrieval: https://colab.research.google.com/github/VectifyAI/PageIndex...

[meander_water]

You could use BM25S [0] instead of rank-bm25 for a nice speedup.

Also, there are tradeoffs associated with using BM25 instead of embedding similarity. You're essentially trading semantic understanding for computational speed and keyword matching.

[0] https://github.com/xhluca/bm25s

[mattnewton]

I recently wrote a short anecdote in a similar vein- in my testing, “agentic” retrieval where you simply pass an annotated list of files to an LLM and ask it which ones it wants to look at is probably better than traditional RAG for small datasets (few hundreds of docs).

I found it was both much simpler and more accurate at the cost of marginally more time and tokens, compared to RAG on embedded chunks with a vector store.

Shameless plug- https://www.matthewnewton.com/blog/replacing-rag

[jarirajari]

I was daunted by the amount of components that goes into a basic RAG. I also ended up starting with "agentic retrieval", which I invented. And it seems that many others have invented same/similar thing. It is just so much easier to start with something simple and improve later on.

[cc_ashby]

Hey Alex, we actually had a very similar idea this morning! I could not find your contact anywhere, would love to chat. This is really cool.

[alexmrv]

Sure! www.calendly.com/growth-kinetics

[rekttrader]

Using a vector db that’s created as a post commithook is probably a reasonable improvement. With context lengths getting bigger natural language is getting easier to work with but your leaving a bunch of gains on the floor without them.

No shade on your project, this is an emerging space and we can all use novel approaches.

Keep it up!

[alexmrv]

ooo nice post-commit creation of the db for the "now" state and links to the files for diff/historical? best of both worlds.

[bob1029]

I think BM25 is the right path for most use cases. I'd reach for a single lucene index here, making the commit hash one of the document's indexed fields. This would make FTS across history nearly instant. I'd store it in an ignored folder right next to .git.

For code search only, BM25 might be a bit overkill and not exactly what you want. FM indexes would be a simpler and faster way to implement pure substring search.

Maybe having both kinds of search at the same time could work better than either in isolation. You could frame them as "semantic" and "exact" search from the perspective of the LLM tool calls. The prompt could then say things like "for searching the codebase use FunctionA, for searching requirements or issues, use FunctionB."

[albertdessaint]

Great idea! How does it compare to Graphiti / Zep? They use real time knowledge-graphs to tackle the problem you are trying to solve https://github.com/getzep/graphiti

[namrog84]

I'm no ai expert so consider my opinion having little value besides casual user. But I really like this. It seems clever and easy for me to grasp some pros for this approach.

I could envision a bunch of use cases about this workikg well. Ive personally encounter scenadios where sometimes the ai gets hung up on irrelevant outdated fact. But could still look up if specifically needed.

I could see even an automated short summary of all history that is outdated being updated in the vector db from this too. So not all context is lost.

Keep up the great work!

[richardblythman]

I think part of the reason i like Claude Code so much for non-technical organizational work, is that it's like a controllable memory with human in the loop (as well as a powerful general agent in the loop). Most of what i do is basically just doc creation and management. "Create this doc using this context and store it in this directory." In ChatGPT, I don't have much control or visibility over the memory.

[jmtulloss]

Seems really interesting! It would be great to see some evals to understand how well this works when stacked up against different approaches and use cases.

[alexmrv]

Keen to do this as well! But couldn't find a good reliable eval for memory creation and retrieval, if you have one you could recommend i'd give it a go asap.

[jerpint]

Hey this is awesome! I built something very similar, context-llemur, it’s a CLI and MCP interface to manage context

https://github.com/jerpint/context-llemur

Major difference is a conversation doesn’t get stored, the LLM (or you) can use the MCP/CLI to update with the relevant context updates

[danshalev7]

Very nice. We tackled the temporal issue "versioned" in your case with a temporal knowledge graph: https://www.falkordb.com/blog/building-temporal-knowledge-gr...

[aszen]

Interesting idea, I like the fact that it doesn't use embeddings, and going into historical memory is an extra step that doesn't impact the speed of resolving the current information

[alexmrv]

exactly!

And you can _choose_ to explore the history, which in the most common case is not even needed.

[entanglr]

I'm also working on something in this space ("git-native memory for AI assistants"), with a completely different approach though. Will have my assistant check out your repo!

[simplecto]

Just had a look at the repo -- I'm curious to know where this goes when you add vectors for semantic / hybrid search.

The use of commit-hooks is also very clever (mentioned here in the replies)

[_pdp_]

I mean these are two completely different approaches for completely different purposes. So saying that git replaces vector database is like saying that that a filing cabinet replaces a search engines.

[nunodonato]

I was working on memory consolidation and memory pruning 3 years ago (with GPT3 Davinci). Glad to see more people are working towards this.

[mjk3026]

Really cool work! This is something I’ve been interested in for a while, and I’m excited to try it out. Cheering you on!

[cc_ashby]

Maybe you can use Mercurial for this? You don’t need the complexity of Git for memory.

[thecopy]

This is cool! Do you have any plans on releasing it as an MCP server? Would love to add this to my mcp gateway service https://mcp-boss.com/ (shameless plug)

[alexmrv]

Would have to be productionized, the current repo is a PoC, but if there is enough interest we could def commit some resources to hardening this and adding things like MCP.