{"id":10551,"date":"2026-08-25T07:20:00","date_gmt":"2026-08-25T05:20:00","guid":{"rendered":"https:\/\/www.lukaswojcik.com\/blog\/?p=10551"},"modified":"2026-10-24T05:04:30","modified_gmt":"2026-10-24T03:04:30","slug":"training-crawlers-and-agentic-browsers-in-ga4-which-machine-traffic-appears","status":"publish","type":"post","link":"https:\/\/www.lukaswojcik.com\/blog\/en\/digital-analytics\/training-crawlers-and-agentic-browsers-in-ga4-which-machine-traffic-appears\/","title":{"rendered":"Training Crawlers and Agentic Browsers in GA4: Which Machine Traffic Appears in Reports"},"content":{"rendered":"

Automated traffic does not split neatly into invisible crawlers and visible AI browsers. Whether a machine visit appears in GA4 depends on the measurement path: what the client executes, which events are sent, and how those events are processed. A page request, a recorded event and a reported session are different observations.<\/p>\n

A training crawler collects content for a particular purpose. An agentic browser performs tasks through a browser. Those descriptions do not, by themselves, establish JavaScript support, identity or visibility in Analytics. The useful starting point is the observed behaviour of a specific service and the site’s actual tracking setup.<\/p>\n

Editorial review: 24 October 2026.<\/small><\/p>\n

JavaScript support is not a dividing line between humans and crawlers<\/h2>\n

A fetcher that only downloads HTML does not execute a browser-side Google tag. That alone cannot produce a browser-tag event. But the broader statement that search crawlers never execute JavaScript is false: Google documents a rendering stage for Googlebot. Rendering still does not guarantee that every resource or analytics request is executed. Google: JavaScript SEO basics<\/a> \u00b7 Google: JavaScript rendering limitations<\/a><\/p>\n

There is also a separate measurement path. A server can send events through the Measurement Protocol, which Google describes as a supplement to tagging. An event therefore does not prove that a human browser produced it, and a page request does not automatically become an event. Google: Measurement Protocol<\/a><\/p>\n

A working browser does not guarantee a reported session<\/h2>\n

An automated browser can execute the same page code as an ordinary browser. Whether it loads the tag and sends an event must nevertheless be observed. A blocked request, a tag trigger that never fires or an interrupted visit can break that path. The presence of a browser window is not proof of successful measurement.<\/p>\n

Google also automatically excludes known bot and spider traffic. Its documentation describes both Google research and the IAB list as inputs, rather than a filter based only on the displayed browser name. The exclusion cannot be disabled and its removed volume is not shown. This is not a promise that all automation is detected. Google Analytics: Known bot-traffic exclusion<\/a><\/p>\n

Three observation layers, three different questions<\/h2>\n
\n\n\n\n\n\n\n\n
Observation<\/th>\nWhat it establishes<\/th>\nWhat it does not establish<\/th>\n<\/tr>\n<\/thead>\n
Request in CDN or server logs<\/td>\nA request reached the logged layer<\/td>\nA human visit or a GA4 session<\/td>\n<\/tr>\n
Analytics request from a browser<\/td>\nA collection attempt occurred<\/td>\nFinal processing or inclusion in a report<\/td>\n<\/tr>\n
GA4 report row<\/td>\nData was included in that report<\/td>\nVerified human identity<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

This distinction also matters when logs disagree. A CDN can answer a request without contacting the origin. Log configuration and caching determine which layer records it. Comparisons need matching timezones, periods and units: HTTP requests cannot be equated with sessions.<\/p>\n

robots.txt expresses rules; it is not an access barrier<\/h2>\n

RFC 9309 explicitly distinguishes crawler rules from access authorization. A rule only affects a client that applies it. JavaScript capability does not determine compliance. Google itself documents different handling for common crawlers and user-triggered fetchers. The behaviour of one service should not be generalized to every AI agent. RFC 9309: Robots Exclusion Protocol<\/a> \u00b7 Google: Verify crawlers and fetchers<\/a><\/p>\n

For access restrictions, the server or CDN must actually enforce a decision. A robots.txt change is not a GA4 report filter and does not remove previously collected events. Its effect on subsequent metrics depends on which requests stop and whether those requests were contributing events.<\/p>\n

Why behavioural clues remain uncertain<\/h2>\n

Repeated actions, short visits and regular timing can motivate an investigation. They can also arise from testing, monitoring or the way an application sends events. No universal rule turns a straight pointer path into a confirmed AI session. Claims about characteristic behaviour require validation against known examples and a measurement of false positives.<\/p>\n

Missing scroll events are particularly easy to misread. Enhanced Measurement records its standard scroll event when 90% vertical depth becomes visible, not at the first movement. A person can click without reaching that threshold. The standard feature does not provide a continuous pointer trace. Google: Enhanced measurement events<\/a><\/p>\n

BigQuery provides event-level detail, not a complete recording of the browser. Google’s schema describes event_timestamp<\/code> as receipt time in microseconds; events sent together can share it. Simple timestamp gaps are therefore not direct measurements of thinking time or proof of automation. Collection and batching must be considered. Google: BigQuery Export schema<\/a><\/p>\n

Cryptographic identity is a different kind of evidence<\/h2>\n

HTTP Message Signatures can authenticate covered message components against a trusted key. Verification also needs appropriate identity binding and replay protections. A valid signature does not establish that the activity is desirable or human. RFC 9421: HTTP Message Signatures<\/a><\/p>\n

As reviewed on 24 October 2026, Web Bot Auth has the working-group draft draft-ietf-webbotauth-httpsig-protocol-00<\/code>, dated 1 September 2026. A working-group Internet-Draft is still not a final RFC, and participation does not cover every automated client. IETF: HTTP Message Signatures for automated traffic<\/a><\/p>\n

Practical workflow with the GA4 traffic review tool<\/h2>\n

The GA4 traffic review tool<\/a> provides a starting point using browser, operating system, country and session source\/medium. Its cues explain which rule matched; they are not bot probabilities. <\/p>\n

Groups below the chosen minimum session count remain in the result but receive no rule flags. The on-screen table displays at most 500 rows; CSV export includes all result rows accepted by the tool. The request is limited to 10,000 rows; analysis stops if the API reports more rows than it returned. A result without flags can therefore also reflect a minimum set too high.<\/p>\n

    \n
  1. Selection of a completed period, with a review of report metadata and collection changes.<\/li>\n
  2. Identification of groups worth checking, including known internal test traffic.<\/li>\n
  3. Comparison of relevant CDN or server evidence over the same period, followed by crawler identity verification using the provider’s documented method.<\/li>\n
  4. Documentation of the conclusion as verified identity, suspected automation or unresolved traffic. These categories should not be merged into a single confirmed bot share.<\/li>\n<\/ol>\n
    \n

    Questions and answers<\/h2>\n

    Does a Chrome row mean that a person visited?<\/h3>\n

    No. The reported browser is context, not identity verification. Automated and human activity can share that label.<\/p>\n

    Does a clean tool result prove that no bots are present?<\/h3>\n

    No. It means that the selected rules did not flag the returned groups. Hidden, excluded or otherwise unrecognized activity is outside that conclusion.<\/p>\n

    Can robots.txt clean historical GA4 data?<\/h3>\n

    No. It communicates access rules to clients that follow them; it does not rewrite Analytics reports.<\/p>\n<\/div>\n

    \n

    Sources<\/h2>\n