Identifying Bots with Digital Signatures: How Web Bot Auth Works

An automated browser can execute JavaScript and therefore appear similar to a human visit in web analytics. This is not inevitable: measurement setup, bot detection and executed actions determine collection. Web Bot Auth adds a cryptographically verifiable identifier for signed requests to those other signals.

The central limit is straightforward: a valid signature demonstrates use of a matching private key for specific message components. It does not establish a verified real-world identity, access permission or harmless behavior.

Three headers with different roles

The foundation is RFC 9421, a published standard for HTTP Message Signatures. Signature-Agent identifies a source for the public verification key. Signature-Input lists covered components and parameters such as creation time, expiry and key identifier. Signature contains the actual signature value.

Ed25519 is one algorithm in use, not a universal restriction of RFC 9421. A verifier needs the algorithm required by the agreed profile, a public key with an authenticated association and exactly the same signature base. The sender signs with a secret private key; the recipient checks the result with the corresponding public key. The signature base is the exact text assembled from the covered components and parameters. Verification does not create a new signature with the public key.

The draft and the product profile differ

As of October 24, 2026, the IETF lists draft-ietf-webbotauth-httpsig-protocol-00, dated September 1, 2026, as a working-group draft. An adopted WG draft therefore exists, but this profile is not yet a completed RFC. The earlier architecture draft has been replaced. The WG draft requires a dictionary form of Signature-Agent; Cloudflare’s currently documented product verification instead requires a single structured string and rejects the dictionary form.

The following lines compare only those formats. This is not a complete signed request: the signature value, time parameters and key material are intentionally absent.

IETF WG draft 00 — dictionary member:
Signature-Agent: sig="https://agent.example"
Covered component: "signature-agent";key="sig"

Cloudflare documented product profile — structured string:
Signature-Agent: "https://agent.example"
Covered component: "signature-agent"

Directory mode:
https://agent.example/.well-known/http-message-signatures-directory
Content-Type: application/http-message-signatures-directory+json

The fixed path applies to directory mode. The current WG draft also describes other key-discovery mechanisms. Integration depends on the supported profile, version and registration requirements. Cloudflare’s Verified Bot process requires registration; publishing an arbitrary key is insufficient.

What successful verification actually connects

A securely resolved HTTPS directory associates a URL with a published key. A matching signature associates that key with the covered components. The signing-key holder and domain administrator need not be the same person or organization. Association with a known company requires additional trusted information.

The key identifier alone is also insufficient: a cache must retain the verified URL-to-key association. Reusing a known key under a newly asserted URL must not create a new identity. Fetching a directory neither proves intent nor replaces local access policy.

Coverage and reuse

A signature covering only @authority and the agent reference does not bind the method, path or message content. While time and other policy checks still pass, copied headers may therefore fit other requests to the same authority (hostname and any specified port). This does not prove that the original signer sent each of those requests.

Additional components such as @method and @target-uri narrow that scope. A signed Content-Digest can cover content; the recipient must also compare the digest with the actual received content. This does not necessarily require buffering the entire content in memory: hashing can proceed incrementally, although content verification finishes only after the final byte.

expires limits the duration of reuse but does not prevent repetition within the window. A nonce is intended for one-time use. It helps prevent copied requests only if the server checks it using an appropriate strategy. Cloudflare currently documents no database check for previously used nonces and recommends short lifetimes.

Verification before an access decision

1. Structured parsing according to the supported profile, followed by checks of required components, parameters and allowed algorithms.
2. Safe key discovery over HTTPS, with permitted destinations, bounded response sizes and timeouts. Client-controlled URLs must not expose internal services. The current WG draft forbids automatically following redirects during discovery.
3. Verification of the URL-to-key association, signature base and cryptographic signature; time checks covering creation, expiry and permitted clock skew.
4. Content-digest and replay checks required by the selected profile, followed by local access, rate-limit and logging rules.

An established library reduces serialization mistakes but does not replace profile checks, key rotation or safe cache rules. Invalid and unsupported evidence remain separate states; neither automatically means human traffic.