AI crawler access
for namecheap.com
namecheap.com permits all 28 tracked AI crawlers. Nothing in its robots.txt restricts training, search indexing or on-demand fetching by the operators tracked here.
That is exactly the median across the 380 sites in this dataset. Resolved from https://namecheap.com/robots.txt on 17 August 2026.
Every crawler, and the rule that decided it
A crawler named directly in its own group is governed by that group. One not named falls back to the wildcard group. Where neither exists the crawler is reported as not listed — no policy has been expressed, and in practice most crawlers will proceed.
| User-agent | Operator | Purpose | Access | Decided by |
|---|---|---|---|---|
OAI-SearchBot | OpenAI | search indexing | Allowed | wildcard policy |
GPTBot | OpenAI | model training | Allowed | wildcard policy |
ChatGPT-User | OpenAI | on-demand fetch | Allowed | wildcard policy |
Claude-SearchBot | Anthropic | search indexing | Allowed | wildcard policy |
ClaudeBot | Anthropic | model training | Allowed | wildcard policy |
Claude-User | Anthropic | on-demand fetch | Allowed | wildcard policy |
PerplexityBot | Perplexity | search indexing | Allowed | wildcard policy |
Perplexity-User | Perplexity | on-demand fetch | Allowed | wildcard policy |
Google-Extended | Gemini training control | Allowed | wildcard policy | |
Bingbot | Microsoft | Copilot search index | Allowed | wildcard policy |
Applebot-Extended | Apple | Apple Intelligence control | Allowed | wildcard policy |
Meta-ExternalAgent | Meta | model training | Allowed | wildcard policy |
Amazonbot | Amazon | model training | Allowed | wildcard policy |
Bytespider | ByteDance | model training | Allowed | wildcard policy |
CCBot | Common Crawl | training dataset | Allowed | wildcard policy |
GoogleOther | research and development fetch | Allowed | wildcard policy | |
Google-CloudVertexBot | Vertex AI on-demand fetch | Allowed | wildcard policy | |
Meta-ExternalFetcher | Meta | on-demand fetch | Allowed | wildcard policy |
MistralAI-User | Mistral AI | on-demand fetch | Allowed | wildcard policy |
cohere-ai | Cohere | model training | Allowed | wildcard policy |
DuckAssistBot | DuckDuckGo | on-demand fetch | Allowed | wildcard policy |
YouBot | You.com | search indexing | Allowed | wildcard policy |
Timpibot | Timpi | training dataset | Allowed | wildcard policy |
AI2Bot | Allen Institute for AI | model training | Allowed | wildcard policy |
Diffbot | Diffbot | structured-data extraction | Allowed | wildcard policy |
ImagesiftBot | ImageSift | training dataset | Allowed | wildcard policy |
PetalBot | Huawei | search indexing | Allowed | wildcard policy |
Omgilibot | Webz.io | training dataset | Allowed | wildcard policy |
What this does and does not tell you
It tells you what namecheap.com states in its robots.txt, which is the file every well-behaved AI crawler consults before fetching. It is a stated preference and a voluntary protocol — not an access control, and not evidence about what any crawler actually did.
It also says nothing about whether the content is useful to an answer engine once fetched. Access is the precondition; extractable passages, visible dates, named authorship and sourced claims decide whether a reachable page is actually cited. That second layer is what the free evidence-readiness report measures.
Finally, robots.txt changes. This page reflects namecheap.com as it was on 17 August 2026. To see the position right now, run it through the live crawler access checker.
Check your own site
The same 28 user-agents, resolved against your live robots.txt, in about five seconds and without an account: AI crawler access checker. If the result is not what you intended, the robots.txt policy builder composes a corrected block, and how to get cited by ChatGPT explains why blocking GPTBot alone does not remove you from ChatGPT answers.
Browse the full study of 380 measured domains, or the hosting and infrastructure category, to see how this compares.
Embed this result
A live badge for namecheap.com, updated whenever this report refreshes. Paste it into a README or site footer:
<a href="https://www.indexhalo.com/ai-crawler-report/namecheap.com/">
<img src="https://www.indexhalo.com/badge/ai-crawler-access/namecheap.com.svg"
alt="AI crawler access for namecheap.com: 100%" height="20">
</a>