Facebook's AI answer pages are shut to ChatGPT's crawler

Each page is an AI-written summary of one topic, built from public Facebook posts by people and pages, according to a write-up by Saul Westbrook on Peripheral Creative. He says the sources include group posts, page posts and comments, some dating back to 2010, and his example is a page on using American Express for travel. He first noticed the pages in June 2026. Glenn Gabe, quoted by Search Engine Roundtable, says Facebook started populating the directory in July 2026.

Westbrook says that as of 3 November Facebook had not spoken publicly about the pages. In June, Meta launched an AI Mode for searching Facebook that answers from public posts, including Groups and Reels, TechCrunch reported. Westbrook's view is that the pages are probably static versions of what that feature generates. Meta has not confirmed it.

Semrush estimates quoted by Westbrook put the pages' UK search traffic at 2,936,009 visits in October 2026, up from 5,445 in May, and worldwide traffic at 40,590,906 visits in October. They are a tool's estimates, and Facebook has published no figures of its own. Westbrook says the pages survived Google's September spam update and that Google's AI Overviews frequently cite them. Gabe reported visibility in AI Overviews, AI Mode and Gemini, and called the pages "Scaled content abuse".

That is a term from Google's spam policies, which define scaled content abuse as "when many pages are generated for the primary purpose of manipulating search rankings and not helping users" and list "Using generative AI tools or other similar tools to generate many pages without adding value for users" as an example. Google has not said whether these pages break that policy, and Search Engine Roundtable quoted no comment from Google or Meta. Its editor, Barry Schwartz, wrote: "I suspect this stuff won't be indexed too much longer, now that it is out there publicly."

Facebook's robots.txt names 35 crawlers and closes its rules with a catch-all, `Disallow: /`, for every other crawler. Six AI crawlers are named and blocked from the whole site: OpenAI's GPTBot, Anthropic's ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended and Amazonbot. OpenAI's search crawler, OAI-SearchBot, and Anthropic's Claude-SearchBot are not named, so the catch-all blocks them too. Googlebot, Bingbot, Applebot and DuckDuckBot are allowed into /fb-answers/. The same file blocks Googlebot and Bingbot from a different folder, /meta-ai-answers/.

What that means depends on each company's published rules. Google's page on AI features says a page needs to be indexed and eligible to be shown in Google Search with a snippet to appear as a supporting link in AI Overviews or AI Mode, and that "There are no additional technical requirements." Bing says the content its crawlers collect can be used for web-grounded answers in Bing, Copilot and Microsoft 365. OpenAI says "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers", though they can still appear as navigational links. Anthropic says blocking Claude-SearchBot may reduce a site's visibility in its search results, and Perplexity says PerplexityBot is the crawler that surfaces and links websites in its results.

Fetchers acting on a single user's request are a separate case: OpenAI says robots.txt rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them. Whether Facebook serves its pages to those fetchers is a different question, which robots.txt does not answer.

Yes, for some brands. In our Google UK reading of 6 October, in which we searched the names of 50 big UK brands three times each, facebook.com was among the sites cited in 11 of the 78 AI Overviews Google showed. Those overviews covered seven brands: Monzo, Zopa, Virgin Media, EE, easyJet, Trainline and Dunelm.

We recorded the domains each overview cited, not the pages, so we cannot say whether any of those citations were Facebook Answers pages rather than a brand's own Facebook page or a public post. The reading does show that Google's AI answers about UK brands already draw on Facebook, and Facebook now publishes AI summaries of what its users say about many topics.

Search Google for `site:facebook.com/fb-answers` with your brand, your products and your category, the search Westbrook used, to see whether a page sums up what Facebook users say about you, and read it for anything wrong or out of date. The pages are built from public posts, so the questions people ask about you in public groups and on your own page, and the answers they get, are the raw material. Facebook has published no way for a brand to correct a page.

Do not rely on Facebook to carry your facts into ChatGPT, Claude or Perplexity, because their search crawlers are not allowed in. Put the facts buyers ask about on pages those crawlers can read: your own site, and the review and comparison sites in your category. Our free Google AI Overview checker runs one search from a clean, logged-out browser in the UK or US and shows whether an overview appeared and which sites it cited, so you can see whether Facebook is one of them. The Google AI Overviews tracker page explains how we measure that over time.

We requested https://www.facebook.com/robots.txt once at 06:08 UTC on 6 November 2026 and received a file with 36 user-agent groups. For each crawler we applied the file's rules to the path /fb-answers/using-american-express-for-travel/, the example page from Westbrook's write-up, taking the most specific matching rule and using the catch-all group for crawlers the file does not name. robots.txt records what Facebook asks crawlers to do. It does not show what any crawler did, or which Facebook pages Google has indexed. Facebook can change the file at any time.

The UK figures come from our 6 October 2026 reading: the names of 50 UK brands searched on Google UK three times each, logged out, with the domains each AI Overview cited recorded. Its CSV is free to download from that reading's page. The page counts, dates and traffic estimates for Facebook Answers are Search Engine Roundtable's, Westbrook's and Semrush's, and we have not re-run them.