AI crawler access
- What we observe
- We fetch your robots.txt and read the published rules for each AI crawler we track.
- What it proves
- Whether your published rules allow or disallow those crawlers.
- What it cannot prove
- Whether a provider actually visits your site, or when.