Control token · Google
Google-Extended
A robots.txt token, not a separate crawler. Disallowing it tells Google not to use content Googlebot already fetched for Gemini training and grounding. It does not affect Google Search ranking or AI Overviews, which follow ordinary Search rules.
- Operator
- Purpose
- Robots.txt control tokens
- Feeds
- Gemini / AI Overviews
- robots.txt token
Google-Extended- Honours robots.txt
- Honours robots.txt
- Documentation
- developers.google.com
Does it appear in server logs?
No. Google-Extended is a token the operator reads in robots.txt, not a user agent that fetches pages. You will never see a request carrying it; the pages are fetched by the operator's ordinary crawler. The string below is what a tool would match if one were ever sent.
Mozilla/5.0 (compatible; Google-Extended)How to block Google-Extended
Add this to the robots.txt at the root of your domain. Robots.txt is advisory: it works only on crawlers that choose to read it. This one is honours robots.txt according to its operator.
User-agent: Google-Extended
Disallow: /
To block every robots.txt control token at once, or to block training while keeping AI search citations, use the robots.txt generator.
How to verify a genuine hit
The user-agent string is self-asserted: anyone can send it from a laptop, and scrapers routinely impersonate well-known crawlers to slip past filters aimed at unknown ones.
There is nothing to verify: no request ever carries this token. Pages are fetched by Googlebot; this token only controls how they may be used.