Tutorial: Measuring Token Consumption per Request and Enforcing a Monthly Budget

A useful API budget connects three records: a reservation before a request, measured usage after the response and a later reconciliation with provider charges. A token total alone is insufficient. Provider, model, service tier, cache class and additional tools can change the price.

1. The provider’s usage schema

In Anthropic Messages, input_tokens describes uncached input, while cache creation and cache reads are separate input categories. Cache creation can be split by lifetime. In OpenAI Responses, input_tokens is the input total; cached tokens and supported cache-write categories are included within it. Adding them to that total would count input twice. These are different accounting structures, not merely different names.

The following invented normalized record uses four non-overlapping classes. The rates are arithmetic examples in USD per million tokens, not a current quotation for a named model. A real adapter must handle the selected endpoint and all applicable cache lifetimes.

from decimal import Decimal, ROUND_CEILING

# Fictional normalized classes and example USD rates per million.
counts = {"input": 12400, "write_5m": 8200, "read": 96000, "output": 1850}
rates = {k: Decimal(v) for k, v in {
    "input": "3.00", "write_5m": "3.75", "read": "0.30", "output": "15.00"
}.items()}
cost_usd = sum(Decimal(counts[k]) * rates[k] for k in counts) / Decimal(1_000_000)
actual_u = int((cost_usd * 1_000_000).to_integral_value(rounding=ROUND_CEILING))
print(cost_usd, actual_u)  # 0.1245, 124500

The exact components are USD 0.0372, 0.03075, 0.0288 and 0.02775, totalling USD 0.1245. Rounding every displayed component first can change the apparent sum. Decimal arithmetic retains the calculation; the reservation ledger below uses integer micro-USD, where USD 1 equals 1,000,000 units. Rounding upward to that unit is a local budgeting policy, not the provider’s invoice rule.

2. Counting before sending and recording after completion

Anthropic’s counting endpoint is free within its rate limits and provides an estimate without applying caching logic. It does not accept every input supported by Messages: the current documentation excludes most server tools, the MCP connector and URL/file-sourced image or document blocks. The model and complete supported request structure matter. A plain-text tokenizer is not a universal estimator for tools, images or message framing.

Streaming usage does not universally appear only in the last event. Anthropic supplies initial usage in message_start and cumulative updates in message_delta; adding every cumulative value would overcount. For example, successive totals of 10 and 20 mean 20 tokens so far, not 30. A supported SDK’s completed-message helper simplifies collection. Interrupted streams and failed connections can leave incomplete usage. Missing data must remain unknown, not become zero.

A durable audit record should link logical operation, unique attempt ID, provider request/response ID when available, timestamps, feature, provider/model, pricing snapshot, reservation and raw usage. Prompts and API keys are unnecessary for this ledger. A retry is a distinct attempt whose possible charge needs its own reservation. A timeout neither proves a charge nor proves that processing stopped; a response header may never have reached the client.

3. A single transaction for the full reservation

The upper bound must cover input, the maximum applicable output, possible cache writes and bounded tool or loop costs. Maximum output alone is not enough. Unknown pricing, unbounded tool loops or a failed ledger write must prevent dispatch. Cache hits should not be assumed when computing the bound.

This executable ledger.py demonstrates admission control for one application, one currency and one UTC calendar-month policy. All callers must use the same local SQLite database. It is not a multi-host database or a complete billing integration. The amount validator caps each value at one million USD in micro-units.

Before the first reservation, initialise(database_path) creates the tables. Output usage can include non-visible reasoning or formatting tokens; visible answer length is not the output meter.

import sqlite3
from datetime import datetime, timezone

class BudgetExceeded(Exception):
    pass

def units(value):
    if type(value) is not int or not 0 <= value <= 10**12:
        raise ValueError("Expected non-negative integer micro-USD, at most 10^12")
    return value

def connect(path):
    db = sqlite3.connect(path, timeout=10, isolation_level=None)
    db.execute("PRAGMA foreign_keys=ON")
    return db

def initialise(path):
    db = connect(path)
    try:
        db.executescript("""
        CREATE TABLE IF NOT EXISTS budgets(
          month TEXT PRIMARY KEY, limit_u INTEGER NOT NULL,
          breached INTEGER NOT NULL DEFAULT 0);
        CREATE TABLE IF NOT EXISTS calls(
          attempt_id TEXT PRIMARY KEY,
          month TEXT NOT NULL REFERENCES budgets(month),
          reserved_u INTEGER NOT NULL, actual_u INTEGER,
          state TEXT NOT NULL CHECK(state IN ('pending','settled')));
        """)
    finally:
        db.close()

def reserve(path, attempt_id, upper_u, limit_u, now=None):
    upper_u, limit_u = units(upper_u), units(limit_u)
    if not isinstance(attempt_id, str) or not attempt_id:
        raise ValueError("Stable, nonempty attempt ID required")
    now = datetime.now(timezone.utc) if now is None else now
    if now.tzinfo is None or now.utcoffset() is None:
        raise ValueError("Timezone-aware timestamp required")
    month = now.astimezone(timezone.utc).strftime("%Y-%m")
    db = connect(path)
    try:
        db.execute("BEGIN IMMEDIATE")
        if db.execute("SELECT 1 FROM calls WHERE attempt_id=?",
                      (attempt_id,)).fetchone():
            db.commit()
            return False  # No second permission to send.
        db.execute("INSERT OR IGNORE INTO budgets(month,limit_u) VALUES (?,?)",
                   (month, limit_u))
        stored_limit, breached = db.execute(
            "SELECT limit_u,breached FROM budgets WHERE month=?", (month,)
        ).fetchone()
        if stored_limit != limit_u:
            raise ValueError("Conflicting limit for this month")
        used = db.execute(
            "SELECT COALESCE(SUM(COALESCE(actual_u,reserved_u)),0) "
            "FROM calls WHERE month=?", (month,)
        ).fetchone()[0]
        if breached or used + upper_u > limit_u:
            raise BudgetExceeded(month)
        db.execute("INSERT INTO calls VALUES (?,?,?,?,?)",
                   (attempt_id, month, upper_u, None, "pending"))
        db.commit()
        return True
    except BaseException:
        db.rollback()
        raise
    finally:
        db.close()

def settle(path, attempt_id, actual_u):
    actual_u = units(actual_u)
    db = connect(path)
    try:
        db.execute("BEGIN IMMEDIATE")
        row = db.execute(
            "SELECT month,reserved_u,actual_u,state FROM calls WHERE attempt_id=?",
            (attempt_id,)
        ).fetchone()
        if row is None:
            raise ValueError("Unknown attempt")
        month, reserved, old_actual, state = row
        if state == "settled":
            if old_actual != actual_u:
                raise ValueError("Conflicting settlement")
            db.commit()
            return
        db.execute("UPDATE calls SET actual_u=?,state='settled' WHERE attempt_id=?",
                   (actual_u, attempt_id))
        if actual_u > reserved:
            db.execute("UPDATE budgets SET breached=1 WHERE month=?", (month,))
        db.commit()
    except BaseException:
        db.rollback()
        raise
    finally:
        db.close()

BEGIN IMMEDIATE acquires the write transaction before reading the total. This prevents two concurrent callers from reserving the same remaining budget. The reservation is committed before the network request; the transaction is not held while waiting for the API. Only a true return from reserve() permits dispatch. False means that the attempt already exists, not permission to send it again. A lock or database error must stop the call.

PSEUDOCODE
validate provider, model, pricing snapshot and complete cost bound
persist a unique attempt ID and audit context
if reserve(database, attempt_id, upper_bound_u, monthly_limit_u) is True:
    send exactly one provider attempt outside the database transaction
    on complete usage:
        persist the original usage and provider identifiers
        calculate cost with the recorded pricing snapshot
        settle(database, attempt_id, actual_cost_u)
    on unknown outcome or unknown cost:
        keep the reservation pending and investigate

The stored limit is fixed for that month. A later breach flag does not cancel requests already admitted.

The flow is pseudocode for the provider adapter: a reviewed pricing snapshot and a complete bound are prerequisites. The actual code above performs no API call. It records a settlement once, accepts an identical repeated settlement and rejects conflicting amounts. If the actual cost exceeds the reservation, it records that fact and blocks further admissions for that month. It cannot undo spend already incurred.

4. Visible records of uncertain attempts

A crash after reservation but before dispatch can leave unused funds reserved. A timeout after dispatch can leave real spend unresolved. Automatically expiring both cases would reopen the budget without evidence. Recovery needs the audit record and provider information; only confirmed no-charge attempts should settle at zero.

The month is assigned when the application reserves the attempt in UTC. Settlement remains attached to that month, even if it arrives later. This is an internal policy, not a statement about invoice timing. Submission, execution, settlement and provider billing periods should remain separate for asynchronous batches and month-boundary reconciliation.

Sources

• Anthropic: token counting
• Anthropic: prompt caching
• Anthropic: streaming messages
• Anthropic: Usage and Cost API
• OpenAI: counting tokens
• OpenAI: prompt caching
• OpenAI: Responses usage
• SQLite: transactions