The Arithmetic Behind an LLM Bill
Contents
The bill for an application that talks to a language model through an interface is easy to estimate and usually estimated wrong. The reason is simple: the estimate is made per request, the payment happens per conversation. One request is a question and an answer. A conversation is that question sent again with every subsequent one.
Everything else – caching, batching, the choice of model – modifies a number that has already been set by that structure, so it is worth having the structure in view before the price list.
Why a Long Conversation Gets Disproportionately Expensive
Models have no memory between calls. Continuing a conversation therefore means sending the whole history so far along again. In the second turn the first question and answer are paid for a second time, in the third turn the first two are, and so on.
The table works this through for a conversation in which the system prompt is 1200 tokens long, every question 150 and every answer 400. Tokens are roughly word fragments; a hundred tokens is about seventy-five words.
| Turn | Input for this turn | Paid in total up to here |
|---|---|---|
| 1 | 1,350 | 1,350 |
| 2 | 1,900 | 3,250 |
| 3 | 2,450 | 5,700 |
| 4 | 3,000 | 8,700 |
| 5 | 3,550 | 12,250 |
| 6 | 4,100 | 16,350 |
| 7 | 4,650 | 21,000 |
| 8 | 5,200 | 26,200 |
After eight turns, 26,200 tokens of input have been paid for, even though the whole conversation is only 5,200 tokens long at the end. And the cost grows faster than the conversation does: twice the number of turns, from four to eight, costs not twice as much but roughly three times.
Of those 26,200 tokens, 21,000 are text the model has already been given – four fifths. Which is the argument for two decisions that have nothing to do with pricing: keeping the system prompt short, and summarising a long conversation instead of carrying it along in full. Both shrink a number that falls due again in every turn still to come.
Four Prices for the Same Token
| Kind | Relative price | What it costs in exchange |
|---|---|---|
| Input | 1× | the baseline every other row is measured against |
| Output | 3× to 5× | nothing – it is simply the expensive half |
| Cached input, read | about 0.1× | an unchanged prefix and a lifetime measured in minutes |
| Cached input, written | about 1.25× | paid once per cache entry, before anything is saved |
| Batch | 0.5× on both sides | latency: an answer within hours rather than seconds |
The cache row is the one with a condition attached. A cache hit requires a prefix that is byte-identical to a previous request, which means a system prompt containing the current timestamp is never cached, and a conversation whose history is edited invalidates everything from the edit onwards. The prefix has to be stable and it has to be at the front.
The write price is why caching a short prompt loses money. Writing costs more than sending, so the entry has to be read several times before it breaks even – roughly two to three hits at the ratios above. A prompt used once per hour with a cache lifetime of five minutes is paid at the write price every single time.
What the Batch Discount Actually Costs
Half price for the same work looks like the easiest decision on the list, and it is – for the work that can take it. A batch request is answered when the provider has capacity, which is a window of hours rather than a promise, and nothing about the request can depend on someone waiting.
That rules out anything interactive and rules in almost everything else: classifying a backlog, generating descriptions for a catalogue, translating an archive, extracting fields from a stack of documents. The useful habit is to sort the work by whether a human is currently looking at the screen, because that question – not the model, not the prompt – decides which of the two prices applies.
Tokens Are Not Words, and Not Equally Cheap in Every Language
A token is roughly four characters of English text, which puts a word at about 1.3 tokens. German and Polish do worse: compounds, inflections and accented characters split into more pieces, and the same sentence translated costs noticeably more tokens than the English it came from.
For a multilingual application this shows up as a per-language cost difference of thirty to fifty per cent for identical content, in both directions – the prompt costs more and so does the answer. It is not a reason to do anything differently; it is a reason to measure per language rather than take one measurement and multiply, which is how a budget ends up short in exactly the two languages nobody tested with.
The Output Side Is the One Nobody Controls
Output is the expensive half and the half that is specified least precisely. max_tokens is a ceiling, not a target; a model asked for a summary will produce whatever length it considers appropriate, and the instruction that shortens it is in the prompt, not in the parameters.
Two habits move the number more than anything else. Asking for a structured answer with short field names rather than prose – a JSON object with three keys costs a fraction of a paragraph saying the same thing. And, where a model supports extended reasoning, remembering that those tokens are billed as output too: a request that returns forty tokens of answer after two thousand tokens of thinking is charged for two thousand and forty.
Related articles
