[asboans]

It would be fun to train an LLM with a knowledge cutoff of 1900 or something

[twh270]

Someone tried this, I saw it one of the Reddit AI subs. They were training a local model on whatever they could find that was written before $cutoffDate.

Found the GitHub: https://github.com/haykgrigo3/TimeCapsuleLLM

[ph4evers]

That’s been done to see if it could extrapolate and predict the future. Can’t find the link right now to the paper.

[creativeSlumber]

This one? "Mind the Gap: Assessing Temporal Generalization in Neural Language Models" https://arxiv.org/abs/2102.01951

[yanis_t]

Not sure we have enough data for any pre-internet date.

[artursapek]

That would be hysterical