We asked AI about 30 brands. 27 were described wrongly

We picked 31 brands across consumer and business categories. One test failed to complete, which left 30 sets of results. For each brand we asked Claude the questions a potential customer would ask, twice. The first run answered from the model's memory, with web search disabled. The second run had live web search on, so the answers were grounded in current pages.

We then extracted every factual claim about the brand from both sets of answers and verified each one against the brand's own website and current public sources. A claim only counted as materially false if it could mislead a buyer: wrong prices, products that no longer exist, features the company does not offer, or wrong facts about availability and ownership. Loose phrasing and harmless trivia did not count.

From memory, 27 of 30 brands were misdescribed in at least one material way. The most common failures were stale pricing, products presented as current that had been discontinued or renamed, and features the brand has never offered. The answers did not read as unsure. They stated old or invented facts in the same confident tone as accurate ones.

Switching web search on improved the picture but did not fix it. 13 of the 30 brands (43 per cent) were still misdescribed. Grounded answers inherit whatever the retrieved pages say, so a stale third-party page produces a confidently wrong answer with a citation attached to it.

We are not naming the affected brands. The point of the study is the rate, not the roll call, and the same test run next month would catch a different set.

A model's memory is a snapshot of the public web from before its training cutoff. Prices change, products get killed and companies pivot after that date, and the model has no way to know. Worse, models weight consensus: if enough old pages repeat a stale fact, the model treats it as true.

Live web search fixes the recency problem but only as far as the pages it reads are right. If the top results for your category are two-year-old comparison articles, the grounded answer will faithfully repeat two-year-old facts about you.

An assistant is probably already describing your brand to buyers, and these numbers say the description is more likely wrong than right if it comes from memory. Three practical steps follow.

First, find out what is actually being said. Run our free AI visibility checker on your domain, and ask the assistants your buyers use about your category directly. Second, make your own site state the current facts plainly: prices, features and availability in clear dated sentences an engine can lift. Third, fix the third-party pages engines lean on, because review sites, directories and comparison articles are where both training data and live answers get their facts. Our ChatGPT visibility tracker page covers how that engine in particular sources what it says.

Fix one: state the correct fact plainly on your own site. The highest-leverage correction is a page on your own domain that states the fact in plain, crawlable text. If your pricing is wrong in AI answers, your pricing page should carry the current numbers as ordinary HTML text, not an image or a figure rendered only by JavaScript. If a dead product keeps surfacing, publish a short product status page that says so directly: 'Acme Legacy was discontinued in March 2025. Its replacement is Acme Cloud.' Assistants doing retrieval lift plain declarative sentences, so give them one to lift.

Fix two: find and correct the sources assistants cite. Retrieval-backed answers cite their sources, which makes the wrong ones findable. Put your buyer questions to the assistants your buyers use, with browsing on, and note the pages each answer actually cites, not the pages it merely searched: only an attributed source shaped the answer, and where a surface shows the searches it ran, keep those in a separate column. The engine-by-engine procedure is in how to find the sources AI uses to recommend your competitors. Then search Google for your business name next to the wrong fact itself, for example 'YourBrand pricing £29', to find any page still repeating it.

Fix three: sweep your own profiles for contradictions. Contradictions in your own scattered footprint cause errors you cannot pin on anyone else. An old LinkedIn tagline or a Google Business Profile showing a previous price range reads, to a retrieval system, as evidence. When sources conflict, the model may pick the stale claim, or blend the versions into something that was never true at all.

Fix four: re-check on a schedule, because models lag. Corrections do not land instantly, and it helps to know which timescale applies to which fix. Changes on your own site and on third-party pages typically show up in retrieval-backed answers within a few weeks, once the pages are recrawled. Errors baked into a model's training memory are different: they persist until the vendor ships a refreshed model, which can take months and is outside anyone's influence. No vendor can get a memory-based error removed on demand, so treat any guarantee of that with suspicion.

One assistant family (Claude) ran the test. Other engines will differ in degree, but the two failure modes, stale memory and inherited errors from retrieved pages, apply to all of them. Verification happened in July 2026 against sources live at the time. Judging what counts as material involves judgement, and we excluded borderline claims rather than inflate the numbers. Thirty brands is a first sample, not a census.

We plan to rerun the test as engines update and to widen the sample, and we will publish the numbers each time, whether they improve or not.

Data availability: this test was run through the Anthropic API, not the consumer Claude app, and the raw answers and scoring sheet were not retained in a form we can publish. The figures above are as recorded at the time. Our next study, The UK AI Shortlist Study, keeps every raw answer, states its method before collection and publishes its data alongside the findings.