IndexHaloAI crawler access:
social and media platforms
Communities, streaming and creative platforms. 26 domains measured; 15 block at least one of the 28 tracked AI crawlers.
15of 26 social and media platforms block at least one AI crawler · median access 93%
Every domain in this category
Sorted by the share of tracked AI crawlers permitted, lowest first. Each report shows all 28 verdicts and the robots.txt rule behind them.
| Domain | Access | Allowed | Blocked | Not listed |
|---|---|---|---|---|
| reddit.com | 0% | 0 | 28 | 0 |
| quora.com | 7% | 2 | 26 | 0 |
| imdb.com | 11% | 3 | 25 | 0 |
| pinterest.com | 11% | 3 | 25 | 0 |
| flickr.com | 25% | 7 | 21 | 0 |
| netflix.com | 32% | 9 | 19 | 0 |
| behance.net | 57% | 16 | 12 | 0 |
| disneyplus.com | 64% | 18 | 10 | 0 |
| hulu.com | 64% | 18 | 10 | 0 |
| vimeo.com | 71% | 20 | 8 | 0 |
| medium.com | 75% | 21 | 7 | 0 |
| pexels.com | 75% | 21 | 7 | 0 |
| metacritic.com | 86% | 24 | 4 | 0 |
| goodreads.com | 93% | 26 | 2 | 0 |
| giphy.com | 96% | 27 | 1 | 0 |
| 500px.com | 100% | 0 | 0 | 28 |
| deviantart.com | 100% | 28 | 0 | 0 |
| dribbble.com | 100% | 28 | 0 | 0 |
| hbomax.com | 100% | 28 | 0 | 0 |
| linkedin.com | 100% | 0 | 0 | 28 |
| rottentomatoes.com | 100% | 28 | 0 | 0 |
| soundcloud.com | 100% | 28 | 0 | 0 |
| spotify.com | 100% | 28 | 0 | 0 |
| twitch.tv | 100% | 28 | 0 | 0 |
| unsplash.com | 100% | 28 | 0 | 0 |
| youtube.com | 100% | 28 | 0 | 0 |
Method and caveats are described on the full study page: one robots.txt fetch per domain, resolved against 28 published AI user-agents, reported as the voluntary policy it is. Check any site yourself with the free crawler access checker.