How to cut LLM API costs without hurting output quality

How LLM API costs are calculated per request

LLM API cost per request = fresh input tokens × input price + cached input tokens × cached input price + output tokens × output price, with any reasoning trace counted as output. Every lever in this guide moves one of those three terms, and none of them needs a change to your serving stack: on Lyceum Serverless Inference there is no base fee to amortise and no idle capacity to fill, so you pay per token, per model.

The order of the levers follows quality risk, not size of saving. The first two leave the model unchanged: a cache hit on an unchanged prompt does not intentionally change the input. Reordering or compressing a prompt can change results, and an output cap can truncate reasoning or the answer. Changing the model comes last, and only with a score to check against.

Here are the list prices we read from each model's own page on 9 October 2026, per 1M tokens:

Two things stand out. Output is the expensive term on every model in the table, between 2.5 and 5 times the input price. And the spread between models is large: the worked example below costs about 20 times more on Kimi-K3 than on Qwen3.8-Flash-Next ($9.00 against $0.45 per 1,000 requests). The catalogue changes, so check the live per-token pricing page before you budget against these numbers.

How to measure LLM cost and output quality before changing anything

Measure two numbers before you change anything: cost per request and a quality score on a fixed evaluation set drawn from real production requests. Make the set big enough to catch the failures you care about and score it once as-is. That score is the number every later change has to defend.

1. Level 1: unit tests as assertions. Format checks, required fields, forbidden strings. They run in seconds on every code change.
2. Level 2: model and human review. Read traces, label a sample good or bad, and use an LLM judge for open-ended answers.
3. Level 3: A/B tests. Reserve these for significant changes, including a candidate model swap.

That cadence is Hamel Husain's three levels of evaluation, and his argument is the one that matters here: he has found that unsuccessful AI products almost always share a common root cause, a failure to create robust evaluation systems. The cost of the levels rises from unit tests to A/B tests, so run the cheap ones constantly and the expensive ones rarely.

Our dashboard gives you a clean cost number per feature, because the Serverless > Usage view shows cost and tokens (input, output, cached, cache share) grouped by model, by member or by API key, over the last 7, 30 or 90 days. Give each feature its own API key. That keeps the before/after split clean: when the support assistant and the code reviewer share a key, no lever shows its real effect.

Put a spend limit on every key while you are at it: our per-key limits keep an experiment's cost under control, because once a key reaches its cap, new requests are refused with HTTP 402 and the error code api_key_spend_limit_reached. We wrote more about spend limits on LLM token usage across a dev team separately.

How prompt caching cuts LLM input costs

Prompt caching cuts input cost by billing a repeated prompt start at the cached input price, and on Lyceum it is automatic: there is nothing to enable. When consecutive requests share an identical leading prefix, the cached portion is billed at the model's cached input price instead of the input price. On Kimi-K3 model that is $0.75 per 1M tokens instead of $3.00, a 75% discount on the cached part. Three practical rules follow from the mechanics:

• Keep the stable part of the prompt at the front: system prompt, tool definitions, retrieved context that rarely changes. Vary only the tail.
• Caching is best-effort. Do not budget on a fixed hit rate; measure your own.
• Verify hits in the response. Cache hits are reported in usage.prompt_tokens_details.cached_tokens, which is absent on a cache miss.

resp = client.chat.completions.create(
    model="moonshotai/kimi-k3",
    messages=messages,
)
details = resp.usage.prompt_tokens_details
cached = (details.cached_tokens or 0) if details else 0  # absent on a cache miss
print(f"cached input tokens: {cached}")

The cost savings are arithmetic, not a promise. Take 1,000 requests where each carries a 2,000-token stable prompt start, 1,000 fresh input tokens and 500 output tokens, on Kimi-K3. These assumptions are illustrative; the prices are the list prices from the table above ($3.00 per 1M input, $0.75 per 1M cached input, $15.00 per 1M output). With no cache hits, the full 3,000 input tokens (the 2,000-token start plus the 1,000 fresh) bill at $3.00 per 1M and the 500 output tokens at $15.00 per 1M, which works out to $16.50 per 1,000 requests. With the stable start cached, those 2,000 tokens bill at $0.75 per 1M instead, and the same mix costs $12.00. That saving assumes the same input and a cache hit. If you reorder the prompt to improve caching, re-run the evaluation because order can affect the answer.

After reordering, trim. LLMLingua (EMNLP 2023) showed that prompt compression can reach up to 20x compression with little performance loss on its benchmarks. Trim retrieved context first, before compressing anything else: stale chunks and duplicated passages are the cheapest tokens to delete because nobody misses them.

How to cap output and reasoning tokens per request

Cap output and reasoning tokens with max_tokens and the reasoning controls, because a reasoning trace counts toward output tokens and is billed at the output price. You control this per request: reasoning_effort: "none" switches reasoning off on models that support it, and instant variants such as z-ai/glm-5.2-instant answer without a reasoning phase. Reasoning is always on for MiniMax-M3, and on GLM-5.3 and GLM-5.3-Flash reasoning_effort: "none" currently moves the reasoning into the visible content instead of removing it; max_tokens is capped at 65,536 per request.

resp = client.chat.completions.create(
    model="moonshotai/kimi-k3",
    messages=messages,
    reasoning_effort="none",   # on models that support switching
    max_tokens=300,
)

In the worked example, capping output at 300 tokens takes the Kimi-K3 mix from $12.00 to $9.00 per 1,000 requests. Cap only where the task tolerates it. A classification step with a fixed JSON schema may tolerate a tight cap after testing; a long-form summary does not. Truncation is a real quality risk, which is exactly what the evaluation set from the section above is for. One failure mode to watch: when the output budget is spent on reasoning before any visible answer is produced, some clients render an empty reply. We documented why that happens and what to check in empty responses on reasoning models.

When to move tasks to a cheaper open model

Move a task to a cheaper open model only when your evaluation set shows the cheaper model holds the score on that task. The same illustrative mix (2,000-token cached start, 1,000 fresh input tokens, 300 output tokens) costs $1.20 per 1,000 requests on MiniMax-M3 model and $0.45 on Qwen3.8-Flash-Next model at list prices read 9 October 2026. On MiniMax-M3 reasoning is always on and the trace counts as output, so those 300 output tokens have to hold the reasoning as well as the answer: measure your real output count before you budget on $1.20. Because every model sits behind one OpenAI-compatible API, the request can use a different model ID, but tool behaviour, output formats, latency and quality still need testing.

The cost savings breakdown below puts every lever of the worked example side by side; the figures are this article's own arithmetic at the list prices above.

How to route simple and complex requests to different models

Route requests by sending light ones to a small model and demanding ones to a larger model, with your own classifier deciding the split. On Lyceum, light requests can go to the lyceum/simple alias and demanding ones to the lyceum/complex alias. If your classifier sends 70% of the worked mix to Qwen3.8-Flash-Next and 30% to MiniMax-M3, the cost lands near $0.68 per 1,000 requests, the illustrative arithmetic of the two list prices above ($0.20 input, $0.05 cached input and $0.50 output per 1M tokens for Qwen3.8-Flash-Next and $0.40 input, $0.10 cached input and $2.00 output per 1M tokens for MiniMax-M3).

The research backs the approach: Hybrid LLM (ICLR 2024) routed queries by predicted difficulty and made up to 40% fewer calls to the large model, with no drop in response quality, and RouteLLM (2024) trained routers on preference data that cut costs by over 2 times in certain cases without compromising response quality. We cover routing requests by difficulty in depth in a separate guide, and choosing the cheapest open model that clears your quality bar in another.

How to prove output quality held after each change

Re-run the same evaluation after every lever and keep only what holds the score. Prompt restructuring, output caps and model swaps can change what the model sees or produces, and the only defence is the fixed set you scored before you started. For open-ended answers, an LLM judge is the practical scorer: the MT-Bench study (NeurIPS 2023) found that strong LLM judges reach over 80% agreement with human preferences, the same level as agreement between humans. Know its blind spots: position bias, verbosity bias and self-enhancement bias are all documented in the same paper, so randomise answer order and do not let the judge grade its own family's output uncritically.

Cap the cost of the test itself. A spend limit on an API key refuses new requests with HTTP 402 once reached, so an A/B test of a candidate model against your evaluation set has a soft spending threshold you set in advance. It is a soft limit: a running request is never cut, so a key can exceed its cap by a few minutes of usage before new requests are refused.

Then repeat the exercise on a schedule. Andreessen Horowitz (a16z) found in 2024 that for an LLM of equivalent performance, the cost is decreasing by 10x every year. The cheap model that lost your evaluation last quarter may clear it this quarter, and the flash-tier model that was a compromise may have a successor. Batch pricing and serving-side techniques such as quantization and speculative decoding also move the price-quality frontier, but as an API buyer you see them only through the per-token price and the score; we compare batch versus real-time inference pricing separately. Either way, the evaluation set, not the invoice, is the asset to maintain.

• Measure cost per request and a quality score before changing anything, with one API key per feature.
• Take the low-risk levers first: cache the stable prompt start, then cap output and reasoning tokens where the evaluation shows the task tolerates it.
• Change the model last, route by your own classifier, and keep a change only if the same evaluation holds.

Our serverless inference is a good fit for this kind of cost work, because open models such as Kimi, GLM, DeepSeek and Qwen run behind one OpenAI-compatible API, billed per token with no base fee, so every lever above shows up directly in your bill. Lyceum is a sovereign European AI inference provider for open-source models: our serverless models run in Paris and Finland, and the prompts and outputs we process are not stored and never used for training. The serverless inference documentation lists the model IDs, the -instant variants and the routing aliases, and the request limits such as the 65,536-token max_tokens cap. Run your own evaluation set against an open model on Lyceum, billed per token, and keep the change only if the score holds. Current list prices per model are on the pricing page.