Google-Extended
Google-Extended is a robots.txt control that lets you opt out of having your content used to train Google's Gemini and Vertex AI models, without affecting Google Search or AI Overviews.
Google-Extended is a training opt-out, not a search setting. It's a token you place in robots.txt to tell Google not to use your content for training its generative models (Gemini, Vertex AI). Critically, it is not a separate crawler and has no effect on how Googlebot indexes you for Search — or on whether you appear in AI Overviews.
This distinction trips people up constantly. Blocking Google-Extended does not remove you from AI Overviews, because those are served through Googlebot, and Google offers no dedicated opt-out from AI Overviews short of blocking Googlebot entirely — which would also delete you from Search. So Google-Extended is purely a content-rights lever about model training; it is not an AEO visibility control. Your actual AI Overview visibility is governed by ordinary crawlability — the access pillar — and the quality of your answer.
Example. A site adds User-agent: Google-Extended / Disallow: / to keep its
content out of Gemini's training set, and its Google Search rankings and AI Overview
appearances are completely unchanged — exactly as intended.
Relevant pillar
Related terms
- GPTBotGPTBot is OpenAI's web crawler that gathers content to train its models, identified by the GPTBot user-agent and controllable through your robots.txt file.
- robots.txtrobots.txt is a plain text file at the root of your domain that tells crawlers which user-agents may access which parts of your site, and is how you allow or block AI crawlers.
- Answer EngineAn answer engine is a search system that responds to a question with a direct, synthesized answer instead of a list of links, usually citing the sources it drew from.