On July 15 we scanned a small European beauty store on Shopify (unnamed here — it’s a prospect, not a client) and scored it 96/100, grade A. The same day, ora’s Agent Readiness Score — the scanner behind the AgentReady open standard, which we implement ourselves — scored the same store 41/100, grade D, near the very bottom of its ~14,000-site index.

Neither scanner is broken. They answer different questions — and if you’re a merchant staring at two contradictory grades, the difference is worth understanding precisely, because the fixes each one implies are different too.

Three reasons the scores diverge

1. They read different pages. Ora’s checks parse the homepage: it found no JSON-LD there, little text without JavaScript, no pricing — all true. Our scanner reads the product page and the Shopify feed, where this store publishes complete Product JSON-LD: price and currency, live availability, brand, SKU, aggregate rating, five structured variant offers. Also true. A Shopify storefront routinely has a JavaScript-heavy homepage and impeccable machine-readable product data one click deeper. An AI shopping agent resolving a product lands on the product page; a generic crawler profiling the company lands on the homepage.

2. They grade against different jobs. Ora’s rubric scores every site on the same generic bar, which includes developer-ecosystem checks: an npm/PyPI SDK, a CLI tool, a developer portal, OpenAPI, a ChatGPT-app listing. A dev-tools company should have those; a skincare shop never will, and it loses 20+ points for it. Our rubric scores only the commerce job: can an agent find, parse and act on the offer. Same site, different exam.

3. They read robots.txt differently. We report “AI agents allowed” because nothing blocks them (permissive by default). Ora reports “no AI crawler directives” because nothing explicitly welcomes them. Both statements describe the same file.

The detail that matters more than either score

The store serves a UCP discovery profile at /.well-known/ucp — checkout, cart, discount, fulfillment capabilities declared. Ora’s Payments category rates that 10/10, “Strong”. But probe the UCP transaction endpoint behind it and it returns 404: the declaration exists, the rail doesn’t. Ora’s own site shows the consequence — its agent-feedback panel sent a real agent to integrate with the store’s catalog, and the agent failed outright (1.8/5, “not recommended”).

That’s the whole lesson of agent readiness in one store: discovery files are declarations; readiness is whether the action behind them works. It’s why Cloudflare’s file-presence checks are a floor, not a finish line, and it applies to every scanner — including ours.

What their report fixed in our scanner

Comparing reports found a real bug on our side: our “content without JavaScript” check counted <script> bodies as visible text, so a page with heavy inline JSON measured as hundreds of kilobytes of “content” (691,376 characters where the honest number is 10,404). Ora’s content-efficiency figure for the same page exposed it; we shipped the fix and regression tests the same day. Two rubrics disagreeing is noise; two rubrics compared honestly is quality assurance.

How to read your own pair of scores

  • A high generic score + low commerce score: agents can profile your company but may stumble buying from you — check your product-page data, variants, and whether declared protocols actually respond.
  • A high commerce score + low generic score (this store’s case): your offer data is solid; your homepage says nothing to machines. The overlapping fix list is short and cheap: Organization + Offer JSON-LD on the homepage, an explicit AI policy in robots.txt, real server-rendered text, and links to your machine resources (llms.txt, feeds) from the page and its HTTP headers.
  • On both rubrics, the rarest points are the same ones: working agent rails. Our reports track UCP, A2A and ARD adoption on every scan — that frontier is where the next distinction gets made.

Sources

Postscript (same day). This comparison changed our own scoring. Under the rubric that produced the 96, working agent rails (a UCP endpoint that answers, an agent card, machine-readable returns and shipping) were tracked but unscored — so mature stores clustered at the top and the grade stopped differentiating. Our scoring matrix v2, shipped today, weighs the transaction rails at 16 points: the same store now grades B (~82) — readable, findable, and honest about what’s still missing before an agent can transact end-to-end. When a comparison exposes your blind spot, the right move is to reprice it, publicly.

A score is an answer to a question — make sure you know which question was asked; the one that decides revenue is whether the agent can finish the purchase.