[mawax]

The comparison misses the mark: unlike humans, LLMs don't consolidate short-term memory into long-term memory over time.

[ako]

That is easily fixed, ask it to summarize it's learnings, store it somewhere, and make it searchable through vector indexes. An LLM is part of a bigger system that needs not just a model, but context and long term memory. Just like human needs to write things down.

LLMs are actually pretty good at creating knowledge: if you give it a trial and error feedback loop it can figure things out, and then summarize the learnings and store it in long term memory (markdown, RAG, etc).

[Q6T46nT668w6i3m]

You’re making the assumption that there’s one, and only one, objective summarization, this is entirely different than “writing things down.”

[imtringued]

This runs into the limitation that nobody has RL'd the models to do this really well.

[griffzhowl]

Over time though, presumably LLM output is going into the training data of later LLMs. So in a way that's being consolidated into the long-term memory - not necessarily with positive results, but depending on how it's curated it might be.

[runako]

> presumably LLM output is going into the training data of later LLMs

The LLM vendors go to great lengths to assure their paying customers that this will not be the case. Yes, LLMs will ingest more LLM-generated slop from the public Internet. But as businesses integrate LLMs, a rising percentage of their outputs will not be included in training sets.

[yellow_postit]

Is this not a tool that could be readily implemented and refined?

[bfuller]

my knowledge graph mcp disagrees