English
Deutsch
PolskiWeb Bot Auth: How Agent Requests Are Signed and What the Signature Proves
Contents
An agentic browser looks like a person in an analytics report because it is a genuine browser engine. The question of who is actually asking can only be estimated from behaviour – or answered by the other side, by signing.
That is what Web Bot Auth is. The mechanism is short, the cryptography unspectacular, and the most interesting thing about it is what it expressly does not prove.
What travels on the wire
The basis is RFC 9421, HTTP message signatures. The operator of an agent generates an Ed25519 key pair, publishes the public half as a JSON web key set at a fixed path on a domain it controls, and signs every outgoing request. Three headers carry the result.
Signature-Agent: "https://example-agent.com"
Signature-Input: sig=("@authority" "signature-agent");
created=1700000000;
expires=1700011111;
keyid="poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U";
tag="web-bot-auth"
Signature: sig=jdq0SqOwHdyHr9+r5jw3iYZH6aNGKijYp/EstF4RQTQdi5N5YYKrD+mCT1HA1nZD…
directory /.well-known/http-message-signatures-directory
content type application/http-message-signatures-directory+json
The bracket after sig= names what is signed. Here it is two
components: the target host and the directory reference.The verifier reads Signature-Agent, fetches the directory from there, looks up the key named by keyid, rebuilds the signature base from the components named in Signature-Input, and recomputes the signature.
What the signature proves
Two things, and both are strong. First: whoever produced it holds the private key matching a public one published under a particular domain. Second: they control that domain, since otherwise they could publish nothing there. Together that yields a verifiable origin, and that is more than a string in the user agent ever was.
Added to this is the substitute for replay protection: expires bounds the validity, and a signature presented after that moment is rejected. A captured request cannot therefore be reused indefinitely.
What it does not prove
The signature covers named components, in the published example the target host and the directory reference. It does not cover the body of the request, and it says nothing about intent. It proves identity, not behaviour.
A valid signature is therefore not a permission. A cleanly signed agent can scrape the same content as an unsigned one, only demonstrably so. The gain lies not in defence but in accountability: knowing which operator asked makes a rule per operator possible – and, in a dispute, makes it provable who was there.
How to verify it
The check is manageable and worth doing as an exercise, because it shows the limits of the scheme directly. Four steps: read Signature-Agent and test it against an expected domain; fetch the directory at the fixed path and find the key for keyid; assemble the signature base from the components named in Signature-Input, in the order given; compute the Ed25519 check and hold expires against the local clock.
The third step is where in-house implementations fail, because the signature base has to match byte for byte. Building it by hand once also explains why the body is not signed along with it: it would have to be buffered in full before the first check could begin.
The state of play, stated honestly
Both specifications are Internet-Drafts – the architecture and the directory, the latter at its fifth revision. An IETF working group has been dealing with it since early 2026, no adopted document exists, and drafts do still change header names and paths.
At the same time several large network and security providers already verify these signatures in production. For a single site that means: understand it, yes; depend on it, not yet. And the sentence to keep in mind while reading the announcements is in the picture as a dashed edge – a signature proves who is asking, and never what they intend.
Questions and answers
Can anyone simply fill the Signature-Agent header with the domain of a well-known operator?
Writing it is easy; getting it to verify is not. The check fetches the key from the directory of exactly that domain, and without the matching private key the Ed25519 verification fails. Because Signature-Agent is itself one of the signed components, it cannot be swapped for another domain afterwards either.
Can a captured, valid signature be reused before expires is reached?
Within the window, in principle yes. expires is only a substitute for replay protection: it limits how long a signature is valid, but it does not stop the same signature from being presented several times during that period. In the example, created and expires are 11,111 seconds apart, a little over three hours.
Then there is the question of what is signed. In the example it is only the target host and the directory reference, not the path, the method or the body. Anyone who knows the three headers of such a request can therefore attach them to a different address on the same host within that period, and the check still succeeds.
In practice this requires the headers to fall into other hands in the first place. Over HTTPS they are encrypted on the wire, but they are visible wherever TLS terminates, at a CDN or a reverse proxy for example, and in logs that record headers. For a rule per operator this means: a valid signature shows that the operator produced these headers, not necessarily that it sent this particular request.
Does Web Bot Auth help to filter agents out of an analytics report?
Only where the request headers arrive. The signature sits in the headers of the HTTP request, and those are seen by the server, a CDN in front of it or the server log. A measurement tag running as JavaScript in the browser cannot read the headers of the request that loaded its own page, so an agentic browser still counts as a person there.
Getting the distinction into the report means making it on the server: running the check there and passing its result on, for instance as a flag the page hands to the tag, or by measuring on the server itself. This presumes that the agent signs at all; unsigned agents stay as invisible as before.
Related articles
