Inference cost per user: how to model AI feature unit economics

Why an AI feature's inference cost per user is variable

An AI feature's inference cost per user is variable because inference is a cost of goods sold (COGS) that follows usage: serverless inference is billed per token with no base fee, so a quiet user costs almost nothing and a heavy one costs real money. Unlike a seat licence, the cost is not fixed by the contract; it is fixed by how your users behave. Teams usually find this out after launch, when the flat price is published and the first heavy users interact several times more than the median user.

The arithmetic fits in one line. Inference cost per active user per month = interactions per user per month × tokens per interaction × blended price per token, where the blended price weights fresh input, cached input and output tokens by each one's own per-token price. Every term in that line is measurable before launch, on a beta cohort or on a comparable feature.

The result has to be a number per user, not per request, because a per-request cost hides the usage distribution that decides whether a flat price holds. Build the model in this order:

1. Count interactions per active user per month from your request log.
2. Count tokens per interaction, including every multiplier: system prompt, retrieved context, retries and tool-call round trips.
3. Price the token mix as a blended price per token, from the fresh input, cached input and output split.
4. Multiply out to a cost per user, then test it at the 90th percentile before you publish a price.

The usage data an LLM cost-per-user model needs

An LLM cost-per-user model needs three inputs, each from a different place: interactions per active user, tokens per interaction, and the token split: how many tokens are fresh input, how many cached input and how many output. Keep them separate, because the levers you pull later act on different terms of the formula.

• Interactions per active user per month: from your own request log, keyed by end-user ID. Count every billed attempt, including retries and tool-call rounds, and map those attempts back to each user interaction.
• Tokens per interaction: from the usage object returned with every completion. Sum across the whole interaction, including tool-call round trips, where each round trip re-sends context, and any billable reasoning tokens.
• The token split: from the same usage object. The cached share is what turns a repeated system prompt from a large cost line into a small one.

How to measure the inputs on Lyceum

Give each AI feature or plan tier its own Lyceum API key. Our dashboard is built for this split: its Serverless > Usage view shows cost and tokens (input, output, cached and cache share) grouped by model, by member or by API key, for the last 7, 30 or 90 days, with CSV export. The split per end user comes from joining that export with your own request log, which knows which user stood behind each key's requests.

Run the export for a full month before you price. A week of beta traffic over-represents the enthusiasts who tried the feature first, and they are precisely the users your price has to survive.

How to calculate inference cost per user

To calculate inference cost per user, price one interaction at the model's three per-token rates, then multiply by interactions per user per month. For each model, cost per interaction = (fresh input tokens × input price + cached input tokens × cached input price + output tokens × output price) ÷ 1,000,000. The worked example uses illustrative assumptions of our own choosing for an LLM feature: a median user running 100 interactions a month, each sending 1,000 cached input tokens, 3,000 fresh input tokens and 800 output tokens. The prices are our list prices, read from the model pages on 9 October 2026: GLM-5.3-Flash model, GLM-5.3 model and Kimi-K3 model, in US dollars per 1M tokens in the table below.

models = {
    # input, cached input, output, USD per 1M tokens (list prices, 9 Oct 2026)
    "GLM-5.3-Flash": (0.20, 0.05, 0.50),
    "GLM-5.3":       (1.40, 0.26, 4.40),
    "Kimi-K3":       (3.00, 0.75, 15.00),
}

cached, fresh, output = 1_000, 3_000, 800

for name, (pin, pcached, pout) in models.items():
    cost = (fresh * pin + cached * pcached + output * pout) / 1e6
    print(f"{name}: ${cost:.5f} per interaction, "
          f"${cost * 100:.3f} per user per month at 100 interactions")

Check the arithmetic on one row using the GLM-5.3 prices: (3,000 × $1.40 + 1,000 × $0.26 + 800 × $4.40) ÷ 1,000,000 = $0.00798 per interaction. The same token mix costs $0.00105 per interaction on GLM-5.3-Flash and $0.02175 on Kimi-K3, so Kimi-K3 lands about 21 times higher. At 100 interactions that is about $0.11, $0.80 and $2.18 per user per month. These are list prices as of 9 October 2026; rerun the calculation with your own token mix and the prices on the model pages before you commit to a number.

How to test gross margin at the 90th percentile (p90)

Test the 90th-percentile (p90) user as a stress case alongside the mean, the full tail and total cohort cost. No single percentile determines whether a flat price works. A plan can lose money on heavy users and still make a margin overall, but you need the full distribution to quantify that risk. The table assumes a p90 user at 3 times the median (300 interactions) and a heavy user at 10 times (1,000). It shows inference cost per user and the share of a $10 plan left before all other costs. These usage assumptions are illustrative, at list prices of 9 October 2026.

Against a $10 monthly plan, the model decides the answer. At Kimi-K3 prices, a heavy user costs $21.75 against $10 of revenue, a loss of $11.75 a month, while each median user leaves about $7.80, so every heavy user needs about one and a half median users to cover it. At GLM-5.3, a heavy user costs $7.98 and leaves $2.02 of the $10, about a 20% margin before any other cost. At GLM-5.3-Flash, a heavy user costs $1.05 and the plan keeps about 90% of its revenue. This holds the token mix constant for comparison. A cheaper model is a saving only if it meets the workload's quality and latency requirements; real token counts can also differ.

For the business as a whole, Andreessen Horowitz (a16z) observed in 2020 that AI companies' gross margins often sit in the 50-60% range, well below the 60-80%+ benchmark for comparable SaaS businesses, and named heavy cloud infrastructure usage and ongoing human support as causes. For a generative AI feature, inference is the part of that cost that scales with the users who use the feature most.

How to enforce a fair-use cap

Enforce a fair-use cap in the billing path, not only in the terms of service. Our API key spend limits are a practical way to do this, because an owner or admin sets the cap once and every new request over it is refused with HTTP 402 and the error code api_key_spend_limit_reached. The limit renews monthly or is a fixed budget, and it is a soft limit read from a ledger settled every few minutes, so a running request is never cut. A key's limit caps everything sent with that key, so one key per plan tier gives a ceiling per tier; a per-user fair-use cap still has to be counted in your own application from your request log. For limits across a whole team, see our article on spend limits on LLM token usage.

How to lower inference cost per user: the levers ranked

Model choice lowers cost most in this fixed-token example, if the cheaper model meets the same quality and latency requirements. Caching repeated prompt starts, capping output length and trimming prompt and context size each save 10 to 19% there, and which of the three saves most depends on the model and your token split:

• Model choice: Kimi-K3 costs about 21 times as much as GLM-5.3-Flash per interaction, a gap larger than every other lever combined.
• Caching repeated prompt starts: prompts still in GPU memory are billed at the model's cached input price, so the 1,000 cached tokens cost $0.00005 instead of $0.0002 per interaction on GLM-5.3-Flash.
• Capping output length: output is about 55% of the Kimi-K3 interaction cost ($0.012 of $0.02175), so a max_tokens cap acts directly on the largest term.
• Trimming prompt and context size: fresh input is the largest share on GLM-5.3-Flash ($0.0006 of $0.00105, about 57%), so retrieval scope and context trimming pay most there.

The table puts a number on each lever, using illustrative changes to the worked example's interaction (our own arithmetic at the list prices above):

The order of the three token levers depends on your own split, which is why the model needs the split and not just a total. Measure first, then pull the lever that touches the biggest term.

How to route light requests to a cheaper model

Route light requests to a cheaper model by sending them to the lyceum/simple alias, which routes to Qwen3.8 Flash Next today, and demanding ones to lyceum/complex alias, which routes to MiniMax M3 today. Your own classifier decides which is which, and each request is billed as the model it reaches (alias pages, read 9 October 2026). At the worked example's token mix, that is $0.00105 against $0.00290 per interaction, so the simple route costs roughly a third of the complex route. The RouteLLM paper (2024) found that routers trained on preference data cut cost "by over 2 times in certain cases" without compromising response quality. Decide what counts as simple by running real requests through both and comparing the answers; the full method is in our article on routing requests by difficulty.

How to price an AI feature: flat, credits, usage-based or hybrid

Price an AI feature using total cohort cost, heavy-user stress cases and the margin needed for other costs. Four structures follow from the usage distribution, each producing a different behaviour:

• Flat with a fair-use cap: simplest to sell, and the cap decides your worst case. Check total cohort margin and the cost of users above p90 before choosing the cap.
• Credits: users prepay a bundle and heavy usage consumes it faster, so the heavy user pays more without a published per-token rate.
• Usage-based: revenue tracks cost, so margin is stable only if the selling price tracks the actual model and token costs, and the buyer carries the variability.
• Hybrid, a base subscription plus usage tiers: predictable for the customer, and the usage tier captures the heavy user. Bessemer Venture Partners' AI pricing playbook (February 2026) puts it directly: hybrid models "win when you're uncertain".

The structure question is related to but distinct from the per-seat question we cover in per-seat licences versus per-token inference. Latency requirements move the price too, which is where batch versus real-time inference pricing comes in. Whichever you pick, the cost side of the model does not change: it is still interactions × tokens × blended price.

When to recalculate AI unit economics

Recalculate AI unit economics after a model or price change, after a prompt restructure, and on a regular cadence as usage drifts. Re-read the 90th percentile each time, not just the average, because the heavy tail can move while the average barely changes. Every billable action creates an entry in the organisation's ledger, which the billing ledger API returns (GET /orgs/{slug}/transactions), so the model can be re-run on current spend rather than on launch-day assumptions.

The tooling is the same per-token metering the platform bills on. Our serverless inference is a good fit for this kind of cost model because we bill per token with no base fee, so the model you build in a spreadsheet runs on the same list prices you would be billed at, and the same usage view tells you what your users actually cost. It is sovereign European AI inference: we serve open models through one OpenAI-compatible API on GPUs in European data centres, and prompts and outputs are processed, not stored and never used for training. Build the model at the median and the 90th percentile, then check it against live per-token pricing. If steady traffic later makes per-token billing the expensive option, that is the point to read when dedicated GPUs beat pay-per-token pricing.