Robots.txt AI Bot Checker
Paste your website URL. We read your live robots.txt and
show whether major AI crawlers are explicitly allowed, blocked with
Disallow: /, or not named in the file.
Analyze your robots.txt
0 of 15 AI bots explicitly allowed
“Allowed” means this bot has its own User-agent block
and is not blocked with Disallow: / (after
Allow: / rules).
| Bot name | Owner | Status |
|---|
Optimize your AI crawlability
Rank Prompt tracks how AI platforms surface your brand and helps you tune content, llms.txt, and technical signals so the right crawlers can find you.
Get startedWhy AI bots matter
Crawler access in robots.txt is the first gate for training and retrieval use cases.
GPTBot & ChatGPT
OpenAI’s crawlers (including GPTBot and related user agents) collect public pages for ChatGPT browsing and model-related use. If they’re disallowed, ChatGPT may be less likely to rely on your site as a source.
ClaudeBot
Anthropic uses ClaudeBot and Anthropic-AI to retrieve web content for Claude. Your robots.txt is the first gate for whether those agents may fetch your pages at all.
PerplexityBot
PerplexityBot powers Perplexity’s live
answers and citations. Blocking it can remove you from those answer panels, sometimes intentionally,
sometimes by accident alongside a broad User-agent: * rule.
This tool only interprets explicit User-agent lines that
match each bot’s token. If a bot isn’t listed in your file, we show “Not specified”; you may still
inherit rules from other blocks depending on crawler behavior.
Go beyond a one-off robots check
Monitor AI visibility over time, compare to competitors, and act on clear recommendations, not just a single static file.