IndexHaloAI crawler access:
hosting and infrastructure
Clouds, CDNs and registrars. 22 domains measured; 0 block at least one of the 28 tracked AI crawlers.
0of 22 hosting and infrastructure block at least one AI crawler · median access 100%
Every domain in this category
Sorted by the share of tracked AI crawlers permitted, lowest first. Each report shows all 28 verdicts and the robots.txt rule behind them.
| Domain | Access | Allowed | Blocked | Not listed |
|---|---|---|---|---|
| aws.amazon.com | 100% | 28 | 0 | 0 |
| azure.microsoft.com | 100% | 28 | 0 | 0 |
| cloud.google.com | 100% | 0 | 0 | 28 |
| cloudflare.com | 100% | 28 | 0 | 0 |
| dnsimple.com | 100% | 28 | 0 | 0 |
| fastly.com | 100% | 28 | 0 | 0 |
| fly.io | 100% | 28 | 0 | 0 |
| gandi.net | 100% | 28 | 0 | 0 |
| hetzner.com | 100% | 28 | 0 | 0 |
| hover.com | 100% | 28 | 0 | 0 |
| kinsta.com | 100% | 28 | 0 | 0 |
| linode.com | 100% | 28 | 0 | 0 |
| namecheap.com | 100% | 28 | 0 | 0 |
| netlify.app | 100% | 28 | 0 | 0 |
| netlify.com | 100% | 28 | 0 | 0 |
| ovhcloud.com | 100% | 28 | 0 | 0 |
| pantheon.io | 100% | 28 | 0 | 0 |
| porkbun.com | 100% | 28 | 0 | 0 |
| railway.com | 100% | 28 | 0 | 0 |
| render.com | 100% | 28 | 0 | 0 |
| vercel.com | 100% | 28 | 0 | 0 |
| wpengine.com | 100% | 28 | 0 | 0 |
Method and caveats are described on the full study page: one robots.txt fetch per domain, resolved against 28 published AI user-agents, reported as the voluntary policy it is. Check any site yourself with the free crawler access checker.