AI crawler access:
government and institutions
Public bodies and international organisations. 18 domains measured; 1 block at least one of the fifteen tracked AI crawlers.
1of 18 government and institutions block at least one AI crawler · median access 100%
Every domain in this category
Sorted by the share of tracked AI crawlers permitted, lowest first. Each report shows all fifteen verdicts and the robots.txt rule behind them.
| Domain | Access | Allowed | Blocked | Not listed |
|---|---|---|---|---|
| who.int | 93% | 0 | 1 | 14 |
| australia.gov.au | 100% | 0 | 0 | 15 |
| belastingdienst.nl | 100% | 15 | 0 | 0 |
| bund.de | 100% | 15 | 0 | 0 |
| canada.ca | 100% | 15 | 0 | 0 |
| cdc.gov | 100% | 15 | 0 | 0 |
| europa.eu | 100% | 15 | 0 | 0 |
| fda.gov | 100% | 15 | 0 | 0 |
| gov.uk | 100% | 15 | 0 | 0 |
| imf.org | 100% | 15 | 0 | 0 |
| irs.gov | 100% | 15 | 0 | 0 |
| oecd.org | 100% | 15 | 0 | 0 |
| sec.gov | 100% | 15 | 0 | 0 |
| service-public.fr | 100% | 15 | 0 | 0 |
| skatteverket.se | 100% | 15 | 0 | 0 |
| un.org | 100% | 15 | 0 | 0 |
| usa.gov | 100% | 15 | 0 | 0 |
| worldbank.org | 100% | 15 | 0 | 0 |
Method and caveats are described on the full study page: one robots.txt fetch per domain, resolved against fifteen published AI user-agents, reported as the voluntary policy it is. Check any site yourself with the free crawler access checker.