top | item 46724343

(no title)

BobbyLLM | 1 month ago

Got it. Answer in 3 parts.

If you're talking about the "Fastrecall" part of it, it's clamped to around 3000 entries. Beyond that...I mean...it's just a JSON file. How much do you hate yourself? lol :)

I use mine as intended (well, by me) as a short term memory store with a TTL of 14 days. Right now it has...350ish stored facts with different expiration dates. You can play around with the settings and max resets (touches), but if ctx > max_items, janitor rolls in and the oldest get pruned anyway.

If you're talking about the chat bloat control part (cut the crap), I set mine to keep the last 2 user/assistant pairs and a soft cap of 2000 characters. That middle part obviously decays during the chat. I like to keep my chatty fast.

If you're asking about the KB (attach) system: generally speaking, the SUMM function seems to generate a summary of upto about 1500 characters. That's what...250 words? Short and sweet...because again, potato pc + smart grep = don't be stupid.

In theory, the KB thing is limited by you SSD and pain tolerance. I have about 1500 items I query against and it's still in the sub-secondish range.

Mentats / vault? Scales via Qdrant.

Does that answer it? None of this shit is enterprise grade; it's purely for personal use.

discuss

order

No comments yet.