GA4 Alerts on Small Numbers: Threshold, Interval and Statistical Noise

An alert on cart events is meant to report when something in the shop breaks. For it to do that, the drop has to be clearly larger than what the numbers do on their own anyway. Only then is the difference significant – large enough not to be mere chance. Where that line gets drawn is the threshold of the alert.

This is exactly where most alerts on quiet shops fail. They are necessarily set so that nothing short of a halving trips them, and so they sit silently through a checkout that is broken for a third of customers. On small numbers, a third is still inside the normal swing.

Which of the two situations a given shop is in is not a matter of judgement. It hangs on a single number: how many events land in one interval, that is, in the time window the alert counts over.

Why Small Numbers Swing More

Counted things always vary, even when nothing has changed. Ten coin tosses rarely give exactly five heads; seven or three come up all the time. A thousand tosses land close to half almost every time. The variation does not go away, then, it simply weighs less and less against the total.

For an alert it is exactly that ratio which decides. At twenty-five events the usual swing is about five, a fifth of the total. At two thousand five hundred events it is about fifty: ten times as much in absolute terms, but only a fiftieth in relative terms. Large numbers are calm, small ones are restless.

From that follows how large a drop has to be at minimum before it is significant and the alert can tell it apart from normal variation at all.

The right-hand column is the whole article. It says what share of the traffic has to disappear before anything happens at all, and on fifty events per interval that share is three quarters. A broken cart button affecting a third of customers stays invisible there. The figures assume a usual threshold and real traffic, which swings more than plain counting does – more on that below.

Why a Lower Threshold Does Not Help

The obvious answer is to lower the threshold so that smaller drops already trip the alert. That does work: the detectable drop falls by about a third. But the number of false alarms rises seventeen-fold at the same time.

With hourly monitoring that means roughly one alarm every other day without anything having happened at all. An alert that cries wolf every other day is muted within a fortnight. After that the shop has no monitoring and a false sense of having some, which is a worse position than the one it started in.

The threshold is a trade between two failures: missing something, or raising an alarm for nothing. As long as the counts are small, that trade is bad at both ends.

A Longer Interval Instead of a Lower Threshold

There is a way that avoids the trade altogether: get more events into one interval by making the interval longer. Four times the events make the alert twice as sensitive, at the same threshold and the same false alarm rate per interval.

Those figures belong to a shop with 4800 events a day, monitored around the clock. The five-minute row is the one usually configured, because five minutes sounds like a fast response – and it is the row that detects nothing at all.

What the longer interval costs is stated in the last column and nowhere else: an outage runs undetected for that long. Four hours is too long for a checkout and entirely reasonable for a nightly data feed. The interval therefore belongs to the thing being watched, not to a general house rule.

Real Traffic Swings More Than Counting Does

Up to here only the variation of counting has been in play. Real traffic also has a daily rhythm, a weekly rhythm and spikes around paydays. Held against a single flat average, all of that counts as variation too: the alert then takes closing time for an outage and the lunchtime peak for a recovery.

That is why the figures in the tables above are worse than plain counting would suggest. It is not a fudge but the price of a baseline that does not know what time it is.

A baseline built from the same hour of the same weekday over the past few weeks takes the rhythm out. That single change does more for sensitivity than any fine-tuning of the threshold – it works out roughly as if the shop had four times the traffic.

When the Numbers Are Simply Too Small

Some shops never reach three-digit counts in any interval short enough to be useful, and no amount of statistics changes that. What works there is to stop counting events and instead put two numbers into a ratio.

Sessions and cart events both fall at night and both rise on a Monday. Their ratio does neither; it stays calm while both individual numbers wander. A threshold on that ratio therefore still works at volumes where a threshold on the plain count is hopeless. A broken cart button moves the ratio immediately while barely touching the number of sessions.

The arithmetic of this article still applies throughout. It just applies to a quantity that is not already buried under the daily rhythm before the alert ever looks at it.

Practical check: a weekday baseline without the inspected day

The GA4 Anomaly Detector now compares a selected reporting day with the same weekday from each of the eight previous weeks. A Thursday is compared with eight Thursdays. The inspected day is excluded from the reference mean and sample standard deviation, so a sharp drop cannot pull its own baseline down.

The tool reads sessions, total users, page/screen views, key events, event count, total revenue and engagement rate through the read-only Data API. The default date is four days ago to leave time for processing, but that delay does not certify that the report is complete. The property’s reporting timezone and the returned metadata remain relevant.

1. Select a completed day and inspect the eight reference dates shown in the result.
2. If a required date is absent from the response, the comparison is withheld. Missing rows are not silently replaced with zeros; a reported zero remains zero.
3. Review the descriptive z-score against the chosen threshold. With a constant baseline, the tool reports either no change or a change from that constant value, without dividing by zero.
4. Download the nine comparison dates as CSV and compare any flagged change with releases, campaigns, holidays and other evidence.

Eight observations are a small reference set. These flags are not calibrated significance tests, and seven concurrent metric checks can generate chance findings. Weekday matching does not remove trends, special events, consent changes or processing delays. No flag proves complete tracking, and a flag does not establish the cause. This is a manual investigation tool, not a scheduled alert service.

Sources: Google runReport reference, metric definitions and NIST guidance on outlier detection and its assumptions.