If you've ever looked at an AI provider's pricing page and seen numbers like "$3 per million input tokens, $15 per million output tokens" and felt no wiser for it, this is the explanation that pricing page assumed you already had.
What a token actually is
A token is a chunk of text — roughly ¾ of a word in English on average, though it varies by language and content. "Calculator" might be one token; a rare word or a chunk of code might be split into several. Providers bill by token count, not by word count or character count, because tokens are what the underlying model actually processes.
Why input and output are priced differently
Every API call has two token counts: input tokens (what you send — your prompt, any context, conversation history) and output tokens (what the model generates back). Output tokens almost always cost more per token than input tokens, often 4–5x more, because generating each output token requires a full forward pass through the model, while processing input tokens can be batched and parallelized more efficiently. A short question that produces a long, detailed answer will cost more than the input length alone suggests.
Cached tokens: the discount most people miss
If you send the same prompt prefix repeatedly — a system prompt, a long document you're asking multiple questions about — most providers let that prefix be cached, and charge a significantly discounted rate (often 90% off) for the cached portion on subsequent calls. This matters enormously for any application that reuses context: a chatbot with a long system prompt, or a tool that answers several questions against the same document, can cut costs dramatically just by structuring calls so the repeated part comes first and stays identical between calls.
The basic cost formula
cost = (input_tokens − cached_tokens) × input_rate + cached_tokens × cached_rate + output_tokens × output_rate
Multiply that per-request cost by your expected requests per day, and you have a daily estimate; multiply by 30 for a monthly one. The two numbers that swing this the most are output length (verbose responses cost more) and whether you're taking advantage of caching for repeated context.
Practical ways to reduce spend
A few changes typically move the needle more than switching providers:
- Cap output length explicitly where the use case allows it — an unconstrained model will often generate more than the answer actually needs.
- Structure prompts so repeated content (system instructions, reference documents) comes first and stays byte-identical between calls, so it's eligible for the cached-token discount.
- Use a smaller, cheaper model for simple classification or extraction tasks, and reserve the most capable (and expensive) model for tasks that actually need its reasoning ability.
- Trim conversation history you send back on each turn — older context that's no longer relevant to the current question is pure cost with no benefit.
Estimate your own usage
Our AI Token Cost Calculator (and the model-specific calculators for OpenAI, Claude, and Gemini) let you plug in your expected input/output token counts and request volume to see per-request, daily, and monthly costs across current model pricing — useful for budgeting before you build, not just after the bill arrives.