Lower your inference bill: token rates, hidden costs and break-even

To lower an inference bill, first work out what one finished task costs and who spends it. Then stop paying twice for text you send on every request. Change how you buy capacity after that, and negotiate rates last.

Why does an inference bill grow faster than usage, and which fix comes first? The usual cause is how many tokens each task sends, and how many of them are sent again on every request.

We are Lyceum, a sovereign European AI inference provider for open-source models. Sovereign means our models run in European data centres: our serverless models run in Paris and Finland. Prompts and outputs are processed, not stored, and never used for training.

Where an inference bill goes: a worked example

A token is the unit a model reads and writes, roughly a short word or part of one. Providers bill three kinds of token separately: input you send, output the model writes, and cached input. Cached input is the part of your prompt that matches the start of a recent request, so it is billed at a lower rate.

Our serverless inference is billed per token, with no base fee. Our pricing page lists input, cached input and output prices for every model, in US dollars per million tokens. On 26 October 2026, GLM-5.3 costs $1.40 for input, $0.26 for cached input and $4.40 for output.

Take a hypothetical support assistant. Each month it sends 200 million input tokens and receives 20 million output tokens. Every request starts with the same long system prompt, the fixed instructions sent before the user's message.

cost = uncached input x input price
     + cached input   x cached input price
     + output         x output price
(token counts in millions, prices in USD per million tokens)

The second case costs $136.80 less, about 37 percent.

The example also shows the next lever. After caching, output is $88 of $231.20, about 38 percent of the bill. Output costs $4.40 per million tokens, over three times the full input price. Each output token you cut now saves more than each extra token you move into the cache, so shorter answers or a smaller model for easy requests come next.

The example favours caching, because it assumes a stable prompt start and steady traffic. It does not show a real customer, a guaranteed hit rate or other providers' prices. A cache hit is best effort, which means it is likely but not guaranteed: we keep recent prompt starts in GPU memory only until newer requests replace them.

Step 1: Price one finished task

Start with what one finished task costs. Finance needs the same number to price one user. Unit economics relates spending to one unit of value. The FinOps Foundation, an industry group that sets practices for managing cloud cost, names cost per token as one such unit, next to cost per request or per workload.

A task that needs three retries or a long answer costs more than its rate suggests. Measure the cost of a finished task, such as one answered ticket or one reviewed pull request.

Our article on inference cost per user builds that model step by step. For coding work, read what coding agents cost per token in production and what AI code review costs per day. For multi-step agents, see agent inference cost optimization.

Step 2: Find which model, member or key spends the money

Then find which model, person or API key spends the money.

Our dashboard shows that breakdown under Serverless → Usage. It lists cost and tokens by model, by member or by API key. The tokens are split into input, output and cached. The cache share is the part of input billed at the cached rate. You can pick the last 7, 30 or 90 days and export a CSV.

An owner or admin can also put a spend limit on an API key: a cap on what that key may spend on serverless inference. It is a soft limit. Spend is settled every few minutes, and a request that is already running is never cut. So a key can go over its cap by a few minutes of usage before it stops. The dashboard has no organisation-wide budget cap, so give each service its own key.

Before you commit to a workload, read how to estimate serverless inference costs. To spread limits across a team, read setting spend limits on LLM token usage.

Step 3: Stop paying twice for repeated prompt text

Next, stop paying for the same text twice. In the example above, input is the largest line on the bill. The lever here is prefix caching. A prefix is the start of a prompt. The KV cache (key-value cache) is the record a model keeps of the tokens it has already read.

Prefix caching stores that record for processed requests and reuses it when a new request starts with the same prefix. It does not change the model's output. On our API, requests that share an identical start, such as a fixed system prompt, have that part billed at the reduced cached rate.

You can see whether it worked. Each response reports cached tokens in `usage.prompt_tokens_details.cached_tokens`. On a cache miss, `prompt_tokens_details` is missing, so read it defensively:

details = resp.usage.prompt_tokens_details
cached = details.cached_tokens if details else 0

Keep the system prompt and tool definitions first. Put changing values, such as timestamps, near the end.

For the other levers, read how to cut LLM API costs without hurting output quality. Routing requests by difficulty sends easy requests to a cheaper model. Batch vs real-time pricing covers work that can wait.

Step 4: Move steady traffic to a dedicated endpoint

Change how you buy capacity only once your traffic is steady. Serverless inference bills each request by its tokens. A dedicated endpoint is a deployment with GPUs reserved for your traffic alone.

The two prices behave differently. Per-token billing follows your traffic, so quiet hours cost nothing. A dedicated endpoint is priced for the capacity you reserve, so the busier you keep it, the less each token costs. Its price is agreed in your contract before launch, so you know the cost up front. That contract price is called commitment pricing.

A replica is a running copy of the model on its GPUs. Under that contract, our dedicated deployments are billed for the time the replicas run, not per request. Replicas keep running, and billing, while they wait for traffic. A deployment with no requests for 1 hour scales down to zero, which stops billing for idle replicas. The first request after that pause is slower, because the replicas have to start again.

To find your break-even, read pay per token vs dedicated GPU inference. Then see how the crossover moves with model size in the token break-even for open models. The break-even volume for self-hosting adds how busy your GPUs are (utilisation) and the software that serves the model (the serving stack) to the count.

Step 5: Compare rates, models and seat licences last

Lower rates come last. Commitment and negotiated discounts lower the rate you pay, not the number of tokens you use.

When you compare providers or models, compare the same model, token types and region.

Start with our cost per million tokens comparison. For one platform's billing in detail, read what you pay per token on AWS Bedrock. For the cost of running in Europe, read the sovereignty premium measured.

For single models, see the cheapest way to run DeepSeek V4 and Qwen3-235B cost per token across providers. For images, read image generation API pricing per image.

If you pay per seat for an AI tool, check that price too. A seat licence charges each user the same, however much they use the tool. Per-token billing charges for what is actually sent and written.

Our article on per-seat licences vs per-token inference works out where the line sits.

Where to start, by situation

If your input is already mostly cached, compare models on price next. Our pricing page lists input, cached input and output prices for every model we serve.