[dougmwne]

I’m not sure what the use case is outside of having real time information about the world as a search replacement.

For example, if a major event or discovery happened, the model would know about it once a critical mass of news stories and discussions had been generated online. You’d probably be looking at a few days before the content accumulated to the point where it would affect the model weights, which encode all human knowledge ever digitized, so some single news article in a training set of trillions of tokens is not enough.

If you want a long term memory of user interactions, long context and RAG seems to do the job nicely as a single fact can be pulled out of a context length of millions of tokens.