Verified crawler identity

Anything can call itself GPTBot, and a bot dodging your policy can call itself Chrome. A policy matched on user-agent alone only governs honest crawlers.

Web Bot Auth fixes that for operators that sign. It's an open standard (HTTP Message Signatures, RFC 9421): a bot operator publishes its public keys at a fixed URL on its own domain:

https://chatgpt.com/.well-known/http-message-signatures-directory

and signs each request with the matching private key. The toll verifies the signature against that directory. A request that verifies is proven to come from whoever controls that domain.

There is nothing to configure. Every request that presents a signature is verified.

What you see

On your Audit page:

• Requests with a verified signature carry a check mark and the operator's directory host (chatgpt.com) in place of their user-agent. The "By agent" table groups them under that identity, so one operator's traffic is one row whatever user-agents it sent.
• A request whose signature fails verification is flagged as a masquerade attempt and counted separately. Unsigned traffic is not flagged.

On your Crawlers settings, operators that publish a key directory carry a Verified identity badge. The badge means the operator's signed traffic is verifiable. It doesn't change your policy by itself.

What it means for your policy

Your allow/block rules match verified identity as well as user-agent:

• A blocked operator can't dodge by changing its user-agent. If your block list matches its verified identity, the block holds even when the request sends a browser UA.
• To allow or block a signing operator outright, add its directory host as a custom crawler rule. For example, a custom rule for chatgpt.com matches every request OpenAI signs, regardless of which product sent it.

Impostors

A crawler that does not sign can still be checked by where it comes from. Google, Bing, DuckDuckGo, Apple, OpenAI, Anthropic, Common Crawl and Mistral publish the addresses their crawlers use. A request that names one of them is compared with that list, and one from outside it is an impostor using the name to read free.

The check is not enforced at first. Your site has to show it can see a caller's real address, and the proof is the real crawler arriving from its published addresses. Once a crawler has done that 20 times within 14 days, its impostors stop reading free. Until then they are recorded and read free, and your audit shows how close each crawler is. A site behind a proxy that hides the caller's address never arms, so a real crawler is never charged because of your setup.

Once a crawler is armed, an impostor is charged like any agent. In your site settings you can turn impostors away instead. You can also switch the check off.

If your site sits behind a proxy, tell your integration which header carries the caller's address:

• WordPress: Content → Trusted client IP header. Behind Cloudflare, use CF-Connecting-IP.
• The SDK: the clientIp option.

Without it, claims show as "not checked" in your audit and nothing is armed.

Some allowlist entries can never be checked, and site settings lists them with the reason: a name naulon does not know, an operator that publishes no address list, or one whose list is not reliable enough to turn a request away. Anyone can send those names and read free.

What it doesn't do

• Crawlers that do not sign classify by user-agent, apart from the address check above. Most crawlers don't sign yet.
• A failed signature never blocks a request by itself. The request is treated as unsigned, your normal policy applies, and the failure is recorded. A bot with a broken clock is not locked out.
• Humans read free. Signatures only identify bots.

Who signs today

Checked against the live directories on 18 August 2026: OpenAI publishes one at chatgpt.com and signs its ChatGPT agent traffic; Meta publishes one at meta.com, covering Meta-ExternalAgent, Meta-ExternalFetcher and Meta-WebIndexer; Ahrefs publishes one at ahrefs.com. Cloudflare verifies the same standard in front of the sites it proxies.

Exa, checked 2 September 2026, publishes its directory at crawler.exa.ai, not on its main domain, and signs every request its search crawler makes. A custom rule must match the host serving the directory, not the company's homepage.

Not signing as of 18 August 2026: Anthropic, Perplexity, Google, Bing, Apple, Amazon, ByteDance and DuckDuckGo, so their crawlers classify by user-agent alone. The Crawlers screen badge lists the operators whose directory we have verified ourselves; an announcement alone does not add one.

naulon's own research agent signs every request, the price probe and the paid read alike, against a key directory we publish. Any Web Bot Auth verifier in front of your site can verify it.