IndexHaloAI crawler access:
e-commerce and retail
Product and marketplace sites in the age of AI shopping answers. 19 domains measured; 5 block at least one of the 28 tracked AI crawlers.
5of 19 e-commerce and retail block at least one AI crawler · median access 100%
Every domain in this category
Sorted by the share of tracked AI crawlers permitted, lowest first. Each report shows all 28 verdicts and the robots.txt rule behind them.
| Domain | Access | Allowed | Blocked | Not listed |
|---|---|---|---|---|
| instacart.com | 11% | 3 | 25 | 0 |
| amazon.com | 14% | 4 | 24 | 0 |
| ebay.com | 71% | 20 | 8 | 0 |
| alibaba.com | 93% | 26 | 2 | 0 |
| chewy.com | 96% | 27 | 1 | 0 |
| aliexpress.com | 100% | 28 | 0 | 0 |
| bestbuy.com | 100% | 28 | 0 | 0 |
| costco.com | 100% | 28 | 0 | 0 |
| deliveroo.co.uk | 100% | 28 | 0 | 0 |
| etsy.com | 100% | 28 | 0 | 0 |
| homedepot.com | 100% | 28 | 0 | 0 |
| ikea.com | 100% | 28 | 0 | 0 |
| justeattakeaway.com | 100% | 28 | 0 | 0 |
| nike.com | 100% | 28 | 0 | 0 |
| target.com | 100% | 28 | 0 | 0 |
| ulta.com | 100% | 28 | 0 | 0 |
| walmart.com | 100% | 28 | 0 | 0 |
| wayfair.com | 100% | 28 | 0 | 0 |
| zalando.com | 100% | 0 | 0 | 28 |
Method and caveats are described on the full study page: one robots.txt fetch per domain, resolved against 28 published AI user-agents, reported as the voluntary policy it is. Check any site yourself with the free crawler access checker.