Lower your inference bill: token rates, hidden costs and break-even

Most of an inference bill is decided by how your requests are built, not by the rate card.

Measure the cost of one finished task, remove the text you pay for twice, and change how you buy capacity last.

The short version
who is writing

We are Lyceum, a sovereign European AI inference provider for open-source models. Our serverless models run in Paris and Finland. We don’t keep your prompts or outputs, and we never use them for training.

worked example

Where an inference bill goes

With 60 % of input from cache, output becomes the largest line: $88 of $231.20, about 38 % of the bill.

Providers bill three kinds of token separately: input you send, output the model writes, and cached input, the part of your prompt that matches the start of a recent request. A hypothetical support assistant sends 200 M input tokens and receives 20 M output tokens a month, every request starting with the same long system prompt.

cost.formula
cost = uncached input × input price
     + cached input   × cached price
     + output         × output price

So what At $4.40 per million, each output token you cut saves more than each extra token you move into the cache: shorter answers or a smaller model for easy requests come next.

Exhibit 2Monthly bill by share of input served from cache$ per month, GLM-5.3 serverless
Input · $1.40 / 1MCached input · $0.26 / 1MOutput · $4.40 / 1M
0 % cached
$368.00
30 % cached
$299.60−19 %
60 % cached
$231.20−37 %
80 % cached
$185.60−50 %
60 % row in detail
input$112.0080 M × $1.40
cached input$31.20120 M × $0.26
output$88.0020 M × $4.40
Source Hypothetical inputs, not a real customer: 200 M input and 20 M output tokens a month. Prices as of 26 Oct 2026. A cache hit is best effort: recent prompt starts stay in GPU memory only until newer requests replace them.
Step 01 / 05 · Measure

Price one finished task

A task that needs three retries or a long answer costs more than its rate suggests.

  • Start with what one finished task costs. Finance needs the same number to price one user.
  • Measure a finished task: one answered ticket, one reviewed pull request.
  • The FinOps Foundation names cost per token as one unit of unit economics, next to cost per request or per workload. [1]
Exhibit 3Each level down includes cost that the level above hides
rate card$ per 1M tokens
per requestincl. retries
per finished taskthe number to manage
Source Lyceum; FinOps Foundation, unit economics [1]
Step 02 / 05 · Measure

Find which model, member or key spends the money

The usage view already splits cost three ways; one API key per service makes it name the owner.

  • Our dashboard shows the breakdown under Serverless → Usage: cost and tokens by model, by member or by API key.
  • Pick 7, 30 or 90 days and export a CSV.
  • A spend limit on an API key is a soft limit. Spend is settled every few minutes and a running request is never cut, so a key can go over its cap by a few minutes of usage.
  • There is no organisation-wide cap, so give each service its own key.
Exhibit 4What the usage view lets you sliceServerless → Usage
Group bySplit intoPeriod
ModelInput7 days
MemberCached input30 days
API key · one per serviceOutput90 days
Export CSV
Source Lyceum dashboard
Step 03 / 05 · Cut waste

Stop paying twice for repeated prompt text

An identical prompt start bills at $0.26 instead of $1.40 per million tokens, if the fixed parts come first.

  • Prefix caching stores the model's record of a prompt start and reuses it when a new request starts the same way. [2]
  • It does not change the output.
  • On our API, the identical start is billed at the cached rate.
Exhibit 5One timestamp at the front sends the whole prompt back to the full rate
✓ fixed parts first
system prompttool definitionsuser messagetime
← $0.26 / 1M (cached)$1.40 / 1M →
✕ timestamp at the front
timesystem prompttool definitionsuser message
whole prompt billed at $1.40 / 1M
Source Lyceum serverless prices as of 26 Oct 2026; vLLM docs, prefix caching [2]
check_cache.pyPython
# missing on a cache miss, so read it defensively
details = resp.usage.prompt_tokens_details
cached = details.cached_tokens if details else 0
Step 04 / 05 · Buy better

Move steady traffic to a dedicated endpoint

Per-token pricing follows your traffic; a dedicated endpoint only gets cheaper per token the busier you keep it.

  • Change how you buy capacity only once your traffic is steady.
  • The two prices behave differently. [4]
Exhibit 6Two ways to buy the same model
serverless
Per token
  • Follows your traffic: quiet hours cost nothing
  • No base fee
  • Best while traffic varies
dedicated endpoint
Per contract
  • Billed for the time replicas run, price agreed before launch
  • The busier you keep it, the less each token costs
  • Scales to zero after 1 hour without requests; the next request is slower
Source Lyceum pricing [4]
Step 05 / 05 · Buy better

Compare rates, models and seat licences last

Commitment and negotiated discounts lower the rate you pay, not the number of tokens you use.

  • When you compare, compare the same model, token types and region. [3]
  • A seat licence charges each user the same however much they use; per-token billing charges for what is sent and written.
Exhibit 7A rate comparison holds only when three things match
  • Same model
  • Same token types input, cached input, output
  • Same region
Source FinOps Foundation, rate optimization [3]

Frequently asked questions

How do I know whether my requests hit the cache?

Read the cached tokens field in each response's usage data, and the cache share in the dashboard's usage view. A missing details field means the request missed the cache.

Your next workload starts here