I develop AI girlfriends.
I've struggled a lot with achieving natural-feeling recall. I've gone through a few iterations, but I'm currently experimenting with a knowledge graph/vector hybrid that uses an LLM to extract facts to build the graph. Both the performance and $ cost really hurt, but it truly does breathe life into AI characters.
I've seen a lot of the commercial products using the latest and most expensive models in the cloud to extract facts. Instead, I fine-tuned a local model on a gpt4-generated dataset, and it works surprisingly well. It will miss some connections but in practice I don't think it will be too noticeable.
tlack|1 year ago
On our [1] chatbots we use one long memories text field per chatbot <-> user relationship.
Each bot response cycle suggests a new memory to add as part of its prompt (along with the message etc)
Then we take that new memory and the existing memories text and feed it to a separate “memory archivist” LLM prompt cycle that’s tasked with adding this new memory and resummarizing the whole thing, yielding a replacement for the stored memories, with this new memory added.
Maybe overly simplistic but easy to manage and pretty inexpensive. The archiving part is async and fast. The LLM seems pretty good sussing out what’s important and what isn’t.
[1] https://Graydient.ai
codekisser|1 year ago
I imagine that your AI chatbots aren't as cheap or performant as they can be with your potentially enormous prompts. Technical details aside, just like when talking to real people, it feels nice when they recall minor details you mentioned a long time ago.
gkorland|1 year ago