[Beltiras]

I'm working on something where I need to basically add on the order of 150,000 tokens into the knowledge base of an LLM. Finding out slowly I need to delve into training a whole ass LLM to do it. Sigh.

[v3ss0n]

https://deepai.org/publication/scaling-transformer-to-1m-tok...

Can this be implemented in current opensource models?

[akvadrako]

Can't you use fine-tuning for this?

A other option is to ask GPT to compress your tokens into a shorter prompt for itself.

[RhodesianHunter]

Or, at this rate, just wait 6 months.

[Zetice]

I don't think this rate is sustainable. [0]

[0] https://www.theverge.com/2023/4/14/23683084/openai-gpt-5-rum...

[Beltiras]

When I would have had to add another 2 batches of ~150,000 tokens.....