[surround]

GPT-5 knowledge cutoff: Sep 30, 2024 (10 months before release).

Compare that to

Gemini 2.5 Pro knowledge cutoff: Jan 2025 (3 months before release)

Claude Opus 4.1: knowledge cutoff: Mar 2025 (4 months before release)

https://platform.openai.com/docs/models/compare

https://deepmind.google/models/gemini/pro/

https://docs.anthropic.com/en/docs/about-claude/models/overv...

[asboans]

It would be fun to train an LLM with a knowledge cutoff of 1900 or something

[twh270]

Someone tried this, I saw it one of the Reddit AI subs. They were training a local model on whatever they could find that was written before $cutoffDate.

Found the GitHub: https://github.com/haykgrigo3/TimeCapsuleLLM

[ph4evers]

That’s been done to see if it could extrapolate and predict the future. Can’t find the link right now to the paper.

[yanis_t]

Not sure we have enough data for any pre-internet date.

[artursapek]

That would be hysterical

[levocardia]

with web search, is knowledge cutoff really relevant anymore? Or is this more of a comment on how long it took them to do post-training?

[mastercheif]

In my experience, web search often tanks the quality of the output.

I don't know if it's because of context clogging or that the model can't tell what's a high quality source from garbage.

I've defaulted to web search off and turn it on via the tools menu as needed.

[MisterSandman]

It still is, not all queries trigger web search, and it takes more tokens and time to do research. ChatGPT will confidently give me outdated information, and unless I know it’s wrong and ask it to research, it wouldn’t know it is wrong. Having a more recent knowledge base can be very useful (for example, knowing who the president is without looking it up, making references to newer node versions instead of old ones)

[WorldPeas]

The problem, perhaps illusory that it's easy to fix, is that the model will choose solutions that are a year old, e.g. thinking database/logger versions from December '24 are new and usable in a greenfield project despite newer quarterly LTS releases superseding them. I try to avoid humanizing these models, but could it be that in training/posttraining one could make it so the timestamp is fed in via the system prompt and actually respected? I've begged models to choose "new" dependencies after $DATE but they all still snap back to 2024

[clickety_clack]

The biggest issue I can think of is code recommendations with out of date versions of packages. Maybe the quality of code has deteriorated in the past year and scraping github is not as useful to them anymore?

[seanw265]

Knowledge cutoff isn’t a big deal for current events. Anything truly recent will have to be fed into the context anyway.

Where it does matter is for code generation. It’s error-prone and inefficient to try teaching a model how to use a new framework version via context alone, especially if the model was trained on an older API surface.

[diegocg]

I wonder if it would even be helpful because they avoid the increasing AI content

[joshuacc]

Still relevant, as it means that a coding agent is more likely to get things right without searching. That saves time, money, and improves accuracy of results.

[alfalfasprout]

It absolutely is, for example, even in coding where new design patterns or language features aren't easy to leverage.

Web search enables targeted info to be "updated" at query time. But it doesn't get used for every query and you're practically limited in how much you can query.

[richardw]

Isn’t this an issue with eg Cloudflare removing a portion of the web? I’m all for it from the perspective of people not having their content repackaged by an LLM, but it means that web search can’t check all sources.

[m3kw9]

Web pages become prompt, so you still need the model to analyze

[stevage]

I've been having a lot of issues with chatgpt's knowledge of DuckDb being out of date. It doesn't think DuckDb enforces foreign keys, for instance.

[roflyear]

Yes, totally. The model will not know about new versions of libraries, features recently deprecated, etc..

[havefunbesafe]

Question: do web search results that GPT kick back get "read" and backpropagated into the model?

[bearjaws]

Falling back to web search is a crutch, its slower and often bloats context resulting in worse output.

[CharlieDigital]

Yes, because it may not know that it needs to do a web search for the most relevant information.

[LeoPanthera]

Gemini does cursory web searches for almost every query, presumably to fill in the gap between the knowledge cutoff and now.

[verytrivial]

I had 2.5 Flash refuse to summarise a URL that had today's date encoded in it because "That web page is from the future so may not exist yet or may be missing" or something like that. Amusing.

2.5 Pro went ahead and summarized it (but completely ignored a # reference so summarised the wrong section of a multi-topic page, but that's a different problem.)

[mock-possum]

I always pick Gemini if I want more current subjects / info

[adhoc_slime]

funny result of this is that GPT5 doesn't understand the modern meaning of Vibe Coding (maximising llm code generation), it thinks it "a state where coding feels effortless, playful, and visually satisfying" and offers more content around adjusting IDE settings, and templating.

[archon810]

And GPT-5 nano and mini cutoff is even earlier - May 30 2024.

[nialv7]

maybe OpenAI have a terribly inefficient data ingestion pipeline? (wild guess) basically taking in new data is tedious so they do that infrequently and keep using old data for training.

[xnx]

Does this indicate that OpenAI had a very long pretraining process for GPT5?

[m3kw9]

Maybe they have a long data cleanup process

[m101]

Perhaps they want to extract the logic/reason behind language over remembering facts which can be retrieved with a search.

[wayeq]

Does the knowledge cut off date still matter all that much since all these models can do real time searches and RAG?

[lurking_swe]

the model can do web search so this is mostly irrelevant i think.

[breadwinner]

That could means OpenAI does not take any shortcuts when it comes to safety.

[dotancohen]

  > GPT-5 knowledge cutoff: Sep 30, 2024
  > Gemini 2.5 Pro knowledge cutoff: Jan 2025
  > Claude Opus 4.1: knowledge cutoff: Mar 2025

A significant portion of the search results available after those dates is AI generated anyway, so what good would training on them do?

[sumedh]

Latest tech docs about a library which you want to use in your code.