security.txt, ads.txt and llms.txt: What a Fixed-Path Check Can Establish

Known paths make selected website information easier to find. A response at such a path does not by itself prove that the expected file is valid. The historical sample below records the output of a bounded checker, not a census of standards compliance.

A registry, conventions and a proposal

RFC 8615 reserves /.well-known/ for application-defined resources. It does not make every resource public without conditions or require a blanket firewall exception. Each format has its own scope and retrieval rules.

The sample also probes root paths. These do not all predate /.well-known/: ads.txt began in 2017 and llms.txt was proposed in September 2024. A sitemap need not be named sitemap.xml. The current llms.txt v2 proposal also allows a subpath such as /docs/llms.txt, so a root-only check cannot establish absence throughout a site.

What the historical sample counted

The saved result is dated 5 September 2026, 18:19:59 +02:00. It contains twenty hosts and thirteen requested paths per host: 260 recorded path results. Redirects may require additional HTTP requests. The table reports the checker’s classification as present; change-password uses its separate redirected classification.

These counts do not establish that age caused adoption. The sample is selected rather than representative, and app-link or advertising files serve particular purposes. A missing file is not automatically a defect.

Four overlaps, not five

Fourteen hosts were classified as having /.well-known/security.txt and five as having /security.txt. Only four belonged to both groups: nytimes.com, bbc.co.uk, gov.uk and onet.pl. The fifth root result, heise.de, had a 404 at the well-known location in this capture.

For web services, RFC 9116 places security.txt under /.well-known/. A root redirect can support older clients without maintaining two copies. A conforming file needs Contact and one Expires timestamp, HTTPS, text/plain and UTF-8. The complete syntax and any Canonical field still need checking; the presence of two field names is not full validation.

A redirect is not proof of a password-change form

The change-password convention directs clients to the account’s password-change page. The snapshot has four redirected classifications, but two ended at status 301 after three recorded hops. The checker had not reached and verified a final form in those cases.

Even a redirect ending in status 200 may lead to a login, homepage or unrelated page. A valid endpoint needs a suitable destination and flow; any redirect is not sufficient evidence.

Refusal and HTML require interpretation

Stack Overflow returned one 418 and seven 429 responses, not proof of a single rule blocking every unknown client. These statuses do not establish whether the requested files existed. Other hosts also returned 403 or 406 responses.

Two results were labelled soft 404 because HTML appeared where another format was expected. That is a rule of thumb based on the response text, not a confirmed diagnosis: challenge pages and legitimate explanatory HTML can resemble this pattern. Content-Type remains relevant to format compliance. Neither a status code nor the first part of a body alone settles every case.

The inspector’s limits

The linked inspector requests thirteen paths, allows up to three redirects and reads up to 204,800 bytes per resource. Its HTML heuristic examines only the first 2,000 characters and excludes the sitemap case. Local fixtures confirmed that plain error text and HTML at sitemap.xml can still be labelled present.

Its parsers provide selected checks, not complete validation. A PGP marker is not signature verification. The link counter misses relative Markdown links, such as /contact instead of a full https:// address. A redirect is not a tested password-change process. Truncated responses and these limits must remain part of the interpretation.

GPC declarations and AI guidance

Two captures contain a positive GPC declaration, but one also records a missing update date. gpc.json is an optional statement of intended support; absence does not prove rejection, and presence does not prove actual processing behaviour. This small sample cannot measure general browser adoption.

llms.txt offers structured guidance and links for agents. It is not an access-control mechanism or a guarantee that a crawler reads the file. ads.txt identifies authorised advertising sellers; a count of records does not independently verify each commercial relationship.

Well-known and Policy Files Inspector