Know which AI crawler arrived—and what job it came to do.
AI Crawl Control turns a generic bot request into a deliberate content decision.
See demand and value
Compare crawlers, operators, paths, bytes transferred, and referral traffic.
WHO IS TAKING?Separate Search, Agent, and Training
In Security Settings, allow useful discovery while treating agent actions and model training as different jobs.
WHAT IS THE PURPOSE?Choose the right outcome
Allow, block, use Redirects for AI Training, deploy AI Labyrinth, or offer Pay Per Crawl and x402 access.
ALLOW · BLOCK · DIRECT · CHARGEContent Signals publish intent; Web Bot Auth verifies legitimate bots; agent-readiness insights and the Analytics API provide evidence. Pay Per Crawl remains a private-beta capability.