We asked AI about 30 brands. 27 were described wrongly

We picked 31 brands across consumer and business categories. One test failed to complete, which left 30 sets of results. For each brand we asked Claude the questions a potential customer would ask, twice. The first run answered from the model's memory, with web search disabled. The second run had live web search on, so the answers were grounded in current pages.

We then extracted every factual claim about the brand from both sets of answers and verified each one against the brand's own website and current public sources. A claim only counted as materially false if it could mislead a buyer: wrong prices, products that no longer exist, features the company does not offer, or wrong facts about availability and ownership. Loose phrasing and harmless trivia did not count.

From memory, 27 of 30 brands were misdescribed in at least one material way. The most common failures were stale pricing, products presented as current that had been discontinued or renamed, and features the brand has never offered. The answers did not read as unsure. They stated old or invented facts in the same confident tone as accurate ones.

Switching web search on improved the picture but did not fix it. 13 of the 30 brands (43 percent) were still misdescribed. Grounded answers inherit whatever the retrieved pages say, so a stale third-party page produces a confidently wrong answer with a citation attached to it.

We are not naming the affected brands. The point of the study is the rate, not the roll call, and the same test run next month would catch a different set.

A model's memory is a snapshot of the public web from before its training cutoff. Prices change, products get killed and companies pivot after that date, and the model has no way to know. Worse, models weight consensus: if enough old pages repeat a stale fact, the model treats it as true.

Live web search fixes the recency problem but only as far as the pages it reads are right. If the top results for your category are two-year-old comparison articles, the grounded answer will faithfully repeat two-year-old facts about you.

An assistant is probably already describing your brand to buyers, and these numbers say the description is more likely wrong than right if it comes from memory. Three practical steps follow.

First, find out what is actually being said. Run our free AI visibility checker on your domain, and ask the assistants your buyers use about your category directly. Second, make your own site state the current facts plainly: prices, features and availability in clear dated sentences an engine can lift. Third, fix the third-party pages engines lean on, because review sites, directories and comparison articles are where both training data and live answers get their facts. Our ChatGPT visibility tracker page covers how that engine in particular sources what it says.

One assistant family (Claude) ran the test. Other engines will differ in degree, but the two failure modes, stale memory and inherited errors from retrieved pages, apply to all of them. Verification happened in July 2026 against sources live at the time. Judging what counts as material involves judgement, and we excluded borderline claims rather than inflate the numbers. Thirty brands is a first sample, not a census.

We plan to rerun the test as engines update and to widen the sample, and we will publish the numbers each time, whether they improve or not.