English
Deutsch
PolskiThe Arithmetic Behind an LLM Bill
Management Summary
Language-model services commonly charge for text processed and generated, measured in small text units called tokens. A conversation can cost more than its visible final answer because earlier messages may be processed again and some work remains unseen. This matters when budgeting for AI assistants or automated tasks. Keeping only useful history, reusing eligible stored input and grouping less urgent work can reduce costs. The saving depends on the service and must still preserve useful answer quality.
Contents
Token-based language-model APIs usually charge for usage in each request. A conversation costs the sum of those requests. Reusing the history can therefore make an early question count as input several times; there is no general flat charge per conversation.
An LLM is a language model that generates text. An API lets software send requests to that model. Input is the material supplied to the model; output is what the model generates. A prompt is the input instruction and context. A cache temporarily keeps reusable results of processing earlier input: writing creates or extends an entry, while reading reuses an existing entry.
A useful estimate separates ordinary input, cache writes, cache reads and output, then adds any tool or service charges. The example below isolates a text conversation with fixed message lengths. It is a calculation model, not a provider invoice or a universal price list.
Why Full History Increases Cumulative Input
An API may require the application to resend history or may manage conversation state on the server. Neither design makes the reused context automatically free. The following example assumes that every request includes the complete earlier conversation, without summarisation, truncation or retrieval-based selection.
The system prompt, which supplies the application’s general instructions, contains 1,200 tokens, each question 150 and each answer 400. A tokenizer splits text into units called tokens, which can be words, word fragments or punctuation. A token is not a fixed number of words. Message formatting, tool definitions, hidden formatting and reasoning tokens (tokens used for internal model processing before the visible answer) are excluded from this simplified calculation. For turn n, input is 1,350 + 550 × (n − 1).
| Turn | Input for this turn | Cumulative input tokens |
|---|---|---|
| 1 | 1,350 | 1,350 |
| 2 | 1,900 | 3,250 |
| 3 | 2,450 | 5,700 |
| 4 | 3,000 | 8,700 |
| 5 | 3,550 | 12,250 |
| 6 | 4,100 | 16,350 |
| 7 | 4,650 | 21,000 |
| 8 | 5,200 | 26,200 |
After eight turns, the cumulative input is 26,200 tokens. The eighth request contains 5,200 input tokens; adding its 400-token answer gives a final conversation of 5,600 tokens including the system prompt. The eight answers total another 3,200 output tokens. The table counts input tokens, not money or total input plus output.
Of the cumulative input, 21,000 tokens repeat earlier input, around 80%. Shorter reusable instructions or a shorter retained history can reduce later input. Summarisation itself can require a paid model call and may lose details, so the relevant comparison includes both that cost and the quality of subsequent answers.
Example Rates, Not a Universal Price List
| Kind | Relative price | Assumption |
|---|---|---|
| Input | 1× | Illustrative reference rate |
| Output | 5× | Example multiplier, not a universal ratio |
| Cached input, read | 0.1× | Example: eligible prefix reused while valid |
| Cached input, written | 1.25× | Example total write rate, not an extra 1.25× surcharge |
| Batch | 0.5× | Example discount; model, endpoint and terms must support it |
Documentation checked on 24 October 2026: Cache terms vary by provider and model. Claude documents 5-minute and 1-hour options; OpenAI GPT-5.6 and later use 30-minute and 12-hour options in Responses. Read rates also vary. The calculation below deliberately keeps fixed example rates; an actual budget needs the selected model’s current pricing and usage categories.
The table uses illustrative multipliers, with ordinary input set to 1. A matching prompt prefix can receive cache-read pricing only when the chosen model’s eligibility, minimum length, cache-boundary and lifetime rules are satisfied. A timestamp can interrupt reuse at the changed point while leaving an earlier stable prefix reusable. Some providers allow an explicit cache boundary marking the end of a reusable input section; others manage caching automatically. Identical request JSON, the structured data format used to send the request, is not the general billing rule.
At a write multiplier of 1.25 and a read multiplier of 0.1, one write plus one complete reuse costs 1.35 instead of 2: a saving of 32.5%. Two or three hits are not required. A short prefix may fail the model’s minimum cache length; an expired entry may require another write. Profitability depends on reuse and tariff, not on prompt length alone.
Batch Processing and Deadlines
Batch processing submits a group of tasks together. The service processes them in the background and makes the results available later. OpenAI documents a 50% discount and a 24-hour completion window; unfinished requests can expire. Anthropic also documents discounted batches and expiry after 24 hours, with many finishing earlier. A batch is therefore neither an immediate-response API nor an unlimited wait without a defined window.
Examples include independent classification tasks, catalogue descriptions and document extraction. Suitability depends on deadlines, supported models, endpoints and features, data limits and retry handling. Whether a person is currently watching a screen is not the billing criterion. Dependent tasks may require separate stages because later inputs are not yet available.
Measuring Token Counts by Language
Tokenization varies with the model vocabulary, language, punctuation and content. Rules such as four characters per token are rough English-text estimates, not a reliable conversion for German or Polish. A translation can use more or fewer tokens depending on the tokenizer and wording.
No fixed 30–50% language surcharge follows from that variation. Representative prompts and answers should be counted separately for each language with the intended model’s tokenizer or token-count endpoint. Actual usage fields then reveal the billed categories. Different token counts are distinct from different prices per token.
Controlling Output and Reasoning
Output limits provide real control: the API-specific parameter sets an upper bound, although not a target length. Names differ, for example max_tokens in Claude Messages and max_output_tokens in OpenAI Responses. A low limit can truncate the answer or consume the allowance in reasoning before visible text appears. Concise instructions and supported effort or verbosity controls address different parts of the problem.
JSON does not inherently cost less than prose: field names, punctuation and the schema can add tokens. A compact response format should be measured against an equally informative alternative. Where reasoning is billed as output, 2,000 billed reasoning tokens plus 40 visible answer tokens yield 2,040 output tokens, provided no other billed output tokens are present. Usage totals, rather than visible text alone, determine the charge.
Questions and answers
How much would a cache save in the conversation from the first table?
Under the stated multipliers, the idealised cache example reduces input cost by about 67.2%. It assumes every prompt is cacheable through its end, each later request reuses the entire preceding request, and the entry remains eligible. Each extension adds the preceding 400-token answer and the next 150-token question. Real thresholds, breakpoints and misses can change this result.
The following calculation uses price units: one unit equals the price of one ordinary input token. These units are neither actual token counts nor currency. Multiplying them by the ordinary input price per token gives the monetary cost:
- Turn 1 writes 1,350 tokens at the write price: 1,350 × 1.25 = 1,687.5.
- Turns 2 to 8 read 21,000 tokens in total, exactly the share the model has already been given: 21,000 × 0.1 = 2,100.
- The same turns write 550 tokens seven times: 3,850 × 1.25 = 4,812.5.
The sum is 8,600 instead of 26,200 price units. At the illustrative output multiplier of 5, the 3,200 output tokens cost 16,000 more in either case: total cost becomes 24,600 instead of 42,200, a saving of about 41.7%. Expiry or a changed prefix can reduce reuse; an unchanged earlier prefix may remain eligible. Summarisation costs and other services are not included.
Does the ratio get even worse for longer conversations?
With full history and these fixed lengths, 16 turns produce 87,600 input tokens, about 3.34 times the eight-turn total. In general, cumulative input is 1,350n + 550n(n − 1)/2. This growth includes a term proportional to the square of the number of turns: each new request repeats more history. That relationship applies to this unbounded-history model, not to every application. Caching, context limits, summarisation and selective history change the actual cost curve.
Related articles
