robots.txt AI training opt-out: what the new token blocks
A draft robots.txt token, __usage_ai-training, says no to AI training without blocking any crawler. Google plans to support it in early 2027.
Research topic
Crawler policy
5 sourced articles on crawler policy, newest first: what changed, what it means for an online store, and what to fix so an AI agent can still find, understand and buy there.
Stealth Bot Prohibition Act: What H.R. 9915 Means for AI Agents
H.R. 9915 would let the FTC fine bots up to $53,000 per violation for hiding their identity, so an honest user-agent could become a legal matter.
Cloudflare's September 15 flip: what our rescan measured
Pre-registered study: every Cloudflare-fronted store in our corpus is rescanned after the September 15 AI-traffic defaults, access-pillar deltas published live.
Cloudflare's September 15 AI Defaults: What Merchants Must Check
Cloudflare's September 15 AI defaults, corrected September 16 from its own notice: Agent blocked on ad pages, Block AI Bots migrated, no Googlebot trap.
Content-Signal in robots.txt: Google Says It Does Nothing
Google told a merchant that Cloudflare Content-Signal robots.txt tags have no effect on any AI crawler, unlike enforced bot-blocking rules.
📡 The agentic-commerce brief
What changed in the protocols — and what it means for your store. Connect it to your agent and it pulls the latest itself; no inbox required.
Add it to your agent (Claude Code, Claude Desktop, Cursor — anything that speaks MCP):
MCP server config
{
"mcpServers": {
"agentready-brief": {
"type": "http",
"url": "https://www.agentready.market/mcp/docs"
}
}
}Then ask it for the AgentReady brief —
it calls get_agentready_brief and reads any entry in full. Prefer a feed
reader? Subscribe by RSS.
No agent? Get it by email instead
Email delivery isn't live yet — nothing will land in your inbox until it is. Leave your address and you'll be on the first issue when it ships.