AI search · Google
Google-CloudVertexBot
Crawls sites on behalf of Vertex AI customers who have built a search or grounding app over specific domains. Only fetches sites those customers configured.
- Operator
- Purpose
- AI search indexers
- Feeds
- Vertex AI grounding
- robots.txt token
Google-CloudVertexBot- Honours robots.txt
- Honours robots.txt
- Documentation
- developers.google.com
What it looks like in server logs
A representative user-agent string. Version numbers change; the Google-CloudVertexBot token does not. Paste your own log line into the user-agent checker to confirm a match.
Mozilla/5.0 (compatible; Google-CloudVertexBot/1.0; +http://www.google.com/bot.html)How to block Google-CloudVertexBot
Add this to the robots.txt at the root of your domain. Robots.txt is advisory: it works only on crawlers that choose to read it. This one is honours robots.txt according to its operator.
User-agent: Google-CloudVertexBot
Disallow: /
To block every ai search indexer at once, or to block training while keeping AI search citations, use the robots.txt generator.
How to allow Google-CloudVertexBot while blocking others
A more specific User-agent group wins over a wildcard. Place this above any User-agent: * block that disallows paths, and Google-CloudVertexBot keeps full access.
User-agent: Google-CloudVertexBot
Allow: /
How to verify a genuine hit
The user-agent string is self-asserted: anyone can send it from a laptop, and scrapers routinely impersonate well-known crawlers to slip past filters aimed at unknown ones.
Google publishes IP ranges for its common crawlers; a hit from outside them is spoofed.