Why your AI doesn’t know what happened last week
We asked six AI models who won the 2026 World Cup, months after the final. Most said it hadn’t been played yet.
On this page
Spain won the 2026 World Cup on July 19. On November 2, we asked six AI models who won, each five times, with search switched off. Twenty-four of the 30 answers said the final had not been played yet.
That is not a bug, and it will not be fixed by the next model. A model’s knowledge stops on the date its training ended. Anything newer reaches it only through a tool, such as web search. If you use AI to check prices, versions, rules or news, you need to know which of the two you just got. This post explains why the gap exists, shows what happened when we tested it, and ends with a three-question check you can use on any answer.
A model’s knowledge stops on a date, and the gap only grows
A language model learns from a huge amount of text, adjusting billions of internal numbers until it can predict that text. When that training ends, the numbers are frozen. Chatting with the model does not change them. The last date covered by that reading is called the knowledge cutoff.
Models do not go on sale the day training stops. Safety testing, tuning and rollout take time. Across 11 models with published dates, the gap between cutoff and launch ran from two to eight months, and the middle of the list is four. So the number to remember is this: a new model’s knowledge is already about four months old on launch day.
The second number matters more, because the clock keeps running after launch. Anthropic promises not to retire Claude Sonnet 5.5 before September 28, 2027. If you are still using it then, its reliable knowledge will be about 15 months old. Local models have no retirement date at all. Llama 4’s pretraining data ends in August 2024, so a copy on your laptop today is working from memories 27 months old.
The last months before the cutoff are the thinnest
A cutoff sounds like a clean line: everything before it known, everything after it unknown. In practice the line is blurry. The world takes time to write about itself, so the final months before a cutoff are covered by far less text than the years before them.
Anthropic publishes this distinction directly. It lists a “reliable knowledge cutoff”, the date through which knowledge is “most extensive and reliable”, and a separate training data cutoff for the broader range of data used. On Claude Sonnet 4.6, the two sit five months apart: August 2025 and January 2026.
Our test showed the same blur. Python 3.14 came out on October 7, 2025, well before the published cutoffs of Claude Sonnet 5.5 (June 2026) and Grok 4.7 (May 2026). Asked for the latest version, Sonnet said 3.13 in all five runs, and Grok said 3.13 in four. Both cutoffs fall at least seven months after 3.14 shipped, yet the older answer won. We can’t see inside the models, but the likely reason is simple: far more text names the older version.
So the cutoff is the outer limit of what a model might know, not a promise. For anything in its final months, expect patchy recall.
A model has no clock, so it assumes no time has passed
The World Cup answers were not made-up facts. None of the 60 no-search answers across our two setups invented a winner (our hallucination explainer covers the cases where models do invent things). The error was about time. In the model’s frozen memory, the final is an upcoming event, and nothing tells it that the calendar has moved.
When we asked “What is today’s date?” with nothing else provided, 17 of 30 answers declined and the other 13 gave a date. All 13 were wrong. DeepSeek V4.1 Flash said May 7, 2026 in all five runs. Consumer apps hide this problem by passing the date in behind the scenes. Anthropic’s documentation says its apps use a system prompt to give Claude “up-to-date information, such as the current date” and that this does not apply to the API. Our test used the API with no such prompt, so an app would behave better on that question alone.
So we added one line, “Today’s date is November 2, 2026.”, and ran the questions again. The date changed the answers a great deal, and it changed what the models knew not at all.
On the World Cup, all 30 answers now said they could not know the result. That is the honest answer. The Python round is more interesting. Twenty-four answers said 3.15 or 3.15.0, which is sensible: Python’s schedule was public, and the models worked forward from it. Claude Sonnet 5.5 labelled that clearly, saying the version “should be” 3.15 and that it could not confirm this.
DeepSeek V4.1 Flash did not. All five runs said some version of “As of November 2, 2026, the latest stable version of Python is 3.15.0.” That sentence reads like something just checked. It is a forecast, and it missed the real answer, 3.15.2, by two patch releases. A date in an answer tells you nothing about where the fact came from.
A model’s own cutoff date is a guess
The obvious shortcut is to ask the model when its knowledge ends. We tried that too, and it does not work.
GPT-6.1 Terra has a published cutoff of June 1, 2026 and said “June 2024” in four of five runs. That is two years off. DeepSeek V4.1 Flash also said “June 2024”, then told us in another round that its Python answer was accurate as of May 2026. Qwen3.8 27B gave five different answers in five runs. Among the models with a published date, only Grok 4.7 came close, saying April 2026 in one run of five against its real May 2026.
Models tend to guess too early. A likely reason, and this is our reading rather than a measured result, is that most text about AI cutoffs that a model trained on was written about older models. Either way, the fix is simple: read the vendor’s model page. OpenAI, Anthropic and xAI publish a date. Where a vendor publishes none, assume several months before launch, per the table above.
Anything newer arrives only through a tool
With web search on, the whole picture flipped. All 36 answers across the two questions were current, and every one returned at least one link. Nothing about the model changed. It simply got to read a fresh page before answering, the same open-book idea behind RAG.
Researchers measured this effect years ago. In 2023, Tu Vu and colleagues built FreshQA, a benchmark of questions whose answers change over time, and collected more than 50,000 human judgments. They found that all models, whatever their size, “struggle on questions that involve fast-changing knowledge and false premises”. Adding current search results to the prompt substantially improved the scores. Models have grown a great deal since, and our test shows that the basic limit has not moved.
One catch for everyday users: search is usually optional, and the model decides. OpenAI’s documentation says the model “can choose to search the web or not”. Google’s says the model “determines if a Google Search can improve the answer”. Anthropic’s docs describe the same behaviour for Claude.
We tested that judgement with a search tool the models could use or ignore, plus today’s date. They did well. On the World Cup and on “Which version of Python should I install for a new project?”, all 36 answers searched. On the boiling point of water, none of the 18 did. That is what you want, though three questions are a small sample, and an app’s own settings can change the behaviour.
Search also costs more, which matters if you pay per call. A bare answer in our test averaged $0.0012. A searched answer averaged $0.026, about 20 times as much. One Claude answer carried 22,967 input tokens, most of them search results, for $0.057. For developers: Anthropic charges $10 per 1,000 searches plus tokens, and Google charges $14 per 1,000 after 5,000 free a month on Gemini 3 and newer. On OpenAI, set tool_choice to required if a question must always search.
Search fixes the staleness, not the reading
A searched answer is a better answer, not a verified one. The model still has to pick good pages and read them correctly. Two of our 36 World Cup answers with search, both from Claude Sonnet 5.5, said the winning goal came in the 105th minute. The match record lists the 106th. The winner, score and scorer were all right, and only opening the source would have caught the slip.
Larger studies found much bigger problems, on harder tasks. In March 2025, Columbia’s Tow Center ran 1,600 queries asking eight AI search tools to identify the source of news excerpts. They answered incorrectly more than 60% of the time, and more than half of the responses from two of the tools cited fabricated or broken links. In October 2025, the European Broadcasting Union and the BBC had journalists review over 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity. They found a significant issue in 45%, serious sourcing problems in 31%, and major accuracy issues, including outdated information, in 20%.
Both studies are more than a year old, and these products have changed since. They still make the useful point: having a link and having the right fact are separate things.
Three questions tell you how far to trust an answer
Most of what you ask does not depend on the date. The boiling point of water has not moved, and a model’s memory is fine for it. The risk sits with anything that changes: prices, versions, laws, office holders, schedules, and last week’s news. For those, run this check.
This matters most for local models, which keep getting older on your disk. A two-year-old open model can still be excellent at writing, coding patterns and summarising your files, because none of that needs last week’s news. Give it search when it does. In CSuite’s chat, the Web toggle lets the model search and read pages from your own computer, and turning it off keeps that chat offline.
If you build on an API, none of this happens for you. Our bare calls show the default: no date, no search, no sources. Put the current date in the system prompt, offer a search tool on any question that can go stale, force it where the answer must be current, and keep the returned links with the answer so someone can check them later. OpenAI’s documentation also requires that citations stay “clearly visible and clickable” when you show web results to users, which helps your readers run the same check.
Our full test took 270 calls and cost $2.19. You can run the short version in your own chat app in a minute: ask about something you know happened last month, then look at whether the reply shows a search. The model’s memory is a sealed box, about four months old on launch day and older every day after. Trust it for what stays true, and ask to see the link for everything else.
