So decide first whether you need a guarantee or only visibility. A guarantee is a capacity purchase. Visibility is a status page and a support line.
Do you have an SLA document you can share? Buyers ask this in procurement, often after a test on a shared API went well. They want that performance in writing. This guide explains what an inference SLA can promise, what it costs and what belongs in the contract next to it.
We are Lyceum, a sovereign European AI inference provider for open-source models. Our serverless models run in Paris and Finland. Serverless means you call a model through our API and pay per token, on GPUs shared with other customers. Prompts and outputs are processed, not stored, and never used for training.
What an AI inference SLA covers
An SLA is a contract with your provider about the level of service. It says what happens when the provider misses it.
The book Site Reliability Engineering splits this into three terms. An SLI (service level indicator) is a measured number, such as request latency or the error rate. An SLO (service level objective) is the target for that number. An SLA is the contract that attaches a consequence to the SLO, most often a rebate or a penalty.
For an LLM API, "down" needs a sharper definition than for a website. A request can fail with an error. It can also succeed so slowly that your app has already given up. Time to first token is how long you wait before the first word of the answer arrives. If your users leave after a few seconds, a slow first token counts as an outage for them. Our guide to time to first token explains what drives it.
The stakes are rising. Deloitte expects inference to take roughly two-thirds of all compute in 2026, up from half in 2025. More of that compute now sits behind a live user session.
Count downtime in failed requests, not minutes
An outage costs you failed requests, and the count depends on when it hits. Take a hypothetical support assistant with 60,000 requests a day. 42,000 arrive between 09:00 and 18:00, and the other 18,000 spread over the remaining 15 hours.
Daytime rate: 42,000 requests / 540 minutes = 77.8 requests per minute
Outage at 11:00: 40 minutes x 77.8 = 3,111 failed requests
Night rate: 18,000 requests / 900 minutes = 20 requests per minute
Outage at 03:00: 40 minutes x 20 = 800 failed requests
The same 40 minutes fail almost four times as many requests at 11:00 as at 03:00. Both outages are 40 of the month's 43,200 minutes. A target written in minutes per month treats them as equal. Your users do not.
The example favours a simple daily curve. Real traffic has spikes inside the busy hours, which make a daytime outage worse. It also ignores retries, which can turn one failed request into several. Our guide to verifying uptime without an SLA shows how to measure this on your own traffic.
So ask any provider how its SLA measures availability. Per request and per endpoint tells you more than one monthly figure for a whole region.
Why pay-per-token APIs rarely come with an SLA
A pay-per-token API serves many customers from one pool of GPUs. The provider cannot promise your throughput without knowing your traffic and setting GPUs aside for it. Throughput here means how many requests or tokens the service handles per second.
GPU supply makes that harder to promise. IDC describes a memory chip shortage, driven by demand from AI data centres, that could persist well into 2027. When capacity is tight, a shared pool is where load shows up first.
Our own serverless inference and smart routing carry no SLA. That means no uptime target, no availability tier and no service credit. Here is what you get instead:
- Live status: our status page shows live service status, latency per model and past incidents.
- Pay per use: per-token billing follows your traffic, so quiet hours cost nothing.
- A way out of single points of failure: one OpenAI-compatible API, so a second provider or model can sit behind the same client code. Our guide to fallback routing that respects EU data rules shows the setup.
That is visibility, not a guarantee. For many products in testing or with uneven traffic, it is enough.
A throughput guarantee needs reserved capacity
A guarantee starts with GPUs reserved for you. On our side, that is a dedicated endpoint. It runs a Hugging Face model on GPUs reserved for your traffic, behind the same OpenAI-compatible API. Each running copy of the model on its own GPUs is called a replica.
The endpoint scales between a minimum and a maximum number of replicas, and you pay for replica time, not per request. We plan the reserved capacity around your production workload and agree the model configuration before launch. For dedicated endpoints and GPU compute, we agree the SLA and availability tier per contract. We usually set the tier during the proof of concept.
One behaviour matters for availability. A dedicated deployment with no requests for one hour scales to zero and pauses. It resumes on the next request. If your traffic has long quiet gaps, plan for that first request in the contract and in your app.
Bring your traffic profile to the talk. The example above gives the shape:
- requests per minute at your busiest hour;
- typical prompt and answer length;
- the latency at which your users give up.
Cost follows the same logic. Per-token billing follows your traffic. A dedicated endpoint is priced for the capacity you reserve, so the busier you keep it, the less each token costs. Over a model's life, serving tends to outweigh training. Luccioni, Jernite and Strubell put parity at hundreds of millions of inferences, a point popular models reach quickly. So the price of guaranteed capacity is a running cost, and worth negotiating with care.
What to check in an inference SLA document
An SLA is only as useful as its definitions. Use this list to read any provider's document, ours included.
| Term | What to ask | Why it matters |
|---|---|---|
| What counts as down | Errors only, or also answers slower than a set latency? | A slow answer can fail your user as surely as an error |
| How it is measured | Per request, per endpoint, per month? | A regional monthly figure can hide your peak-hour outage |
| Exclusions | Maintenance windows, scale-from-zero, traffic above the reserved capacity? | Each exclusion is downtime the SLA does not count |
| Consequence | Service credit, penalty or exit right, and how you claim it | Without a consequence it is an SLO, not an SLA |
| Capacity | Number of replicas, what happens above the maximum | A throughput promise only covers reserved capacity |
| Support | Who answers, through which channel, within what time | Minutes matter while users are failing |
| Reporting | A public status page and incident history | You need evidence to claim a credit |
Our business customers get a direct line to the engineers who run the platform, with contractual response times. Live status sits on our status page.
Put data terms in the same contract
An availability promise and a data promise belong in one place: the contract. A marketing page can change, and a signed agreement does not.
Our GPUs run in European data centres. Prompts and outputs are processed, not stored, and never used for training. The DPA (data processing agreement) is part of our terms. Our ISO 27001 audit is in progress. The data centre operators hold ISO certifications at facility level.
The companies that run parts of our service for us are listed in our subprocessor list, which is available on request. Our guide to subprocessors in the DPA explains what to ask for. Our guide to zero data retention shows how to check how prompts are handled.
Start on our pay-per-token API while your traffic is still taking shape. When you need the guarantee in writing, talk to our engineers about a dedicated endpoint and its SLA.
Frequently asked questions
Do you publish an uptime figure?
No. We publish live status, latency per model and past incidents on status.lyceum.technology, and we quote no uptime percentage. For dedicated endpoints, the availability tier is part of the contract.
Can I get an SLA on your pay-per-token API?
No. Our serverless inference and smart routing carry no SLA and no service credit. If you need a guarantee, a dedicated endpoint is the route, with the SLA agreed per contract.
Do I need a long contract for a dedicated endpoint?
No long-term contract is required. Commitment pricing for a dedicated endpoint is agreed per contract, so you know the cost before launch. Our path is to pay per token as traffic varies, then reserve GPUs when it steadies.