User-agent checker
Is this user agent an AI bot?
Paste a user-agent string from your server logs. Get back whether it is an AI crawler, a search engine, a scraper, a monitor or a real browser, and for AI crawlers, which product a crawl feeds and how to block it.
Result
AI crawler: Meta-ExternalAgent
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)
- Operator
- Meta
- Purpose
- Training crawlers
- Feeds
- Meta AI
- Honours robots.txt
- Honours robots.txt
Meta's crawler for training AI models and improving its products. Distinct from facebookexternalhit, which only renders link previews. The older FacebookBot token served a similar role.
Meta-ExternalAgent: block, allow and verifyTry an example
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot
- Mozilla/5.0 (compatible; ClaudeBot/1.0; [email protected])
- Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
- Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; [email protected])
- Mozilla/5.0 (iPhone; CPU iPhone OS 17_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Mobile/15E148 Safari/604.1
- python-requests/2.32.3
Common questions
How does the checker decide whether a user agent is a bot?
Two layers. First a table of named signatures: AI crawlers, search engines, SEO scrapers, monitors, security scanners and HTTP libraries. Then a set of structural checks for automation that does not name itself, such as a WebKit string with no Safari or Chrome token. The structural rules are deliberately conservative so a real browser is never misfiled.
Is the string I paste stored anywhere?
No. It is read from the URL, classified while the page renders and discarded. The result URL itself is shareable, which is the point: you can link a colleague to the exact classification of a log line.
Why does a real browser show no bot match?
Because it is not one. A Chrome, Safari, Firefox or Edge string returns browser, operating system and device class instead. If a browser string does get flagged, the structural rule that caught it is a bug: email the string to [email protected].
Can a bot fake a browser user agent?
Yes, and many do. A user-agent check catches honest crawlers and careless scripts, which is most automated traffic by volume. Determined scrapers need behavioural or network-level detection. That is why this tool reports what the string claims, and the crawler directory explains how to verify a claim against published IP ranges.