The image is rarely the hard part. The scaling rules and the way you are billed decide whether an endpoint pays off. So compare the hours of running copies your traffic needs with your token bill before you build anything.
Teams with a container ready often ask us the same thing. Can I hand over a Docker image and get an autoscaling API back, or is there a shorter path for my model? It comes up because a container feels like the safe unit to hand over. For most models it is one step more than you need.
What a platform needs from your Docker image
Any platform that turns a container into an API endpoint needs four things from your image. A container is a packaged copy of your code and its dependencies, and the image is the file it starts from. The platform starts it without knowing your code. So the image has to say how to start, where to listen and how to show that it works.
| What the platform needs | Dockerfile instruction | Why it matters |
|---|---|---|
| A server that starts on its own | ENTRYPOINT | The platform runs your container as one executable, with no command typed by you. |
| A known port | EXPOSE | It documents the port your server listens on, but it does not publish it. The platform maps it. |
| A health check | HEALTHCHECK | The platform needs a test that tells a working server from a hung one. |
| GPU access | none, the host provides it | The NVIDIA Container Toolkit on the host lets a container run GPU work. |
Docker's reference describes all three instructions. ENTRYPOINT configures a container that will run as an executable. EXPOSE only documents the port, so a platform cannot route traffic until it knows that number. HEALTHCHECK tells Docker how to test that the container is still working.
The health check matters most of the four. Docker's own example is a web server stuck in an infinite loop: the process still runs, but it answers nothing. Without a check, a platform keeps sending requests to that copy.
GPU access is the fourth need, and the image cannot provide it alone. The NVIDIA Container Toolkit is a set of libraries and tools that let you build and run GPU-accelerated containers. It sits on the host, so your platform has to run it for you.
How autoscaling and scale to zero decide your bill
Autoscaling adds or removes copies of your server as load changes. Kubernetes calls this horizontal scaling: running multiple instances of your app. Each copy that serves your model is a replica, and you pay for every replica that runs.
The rule behind the count is a ratio. The Kubernetes Horizontal Pod Autoscaler sets the replica count from the ratio between the current and the desired value of a metric. For requests, it reads like this, where ceil means round up:
replicas = ceil(current requests per second / requests one replica can hold)Here is the hypothetical example we carry through the article. A support assistant gets 40,000 requests a day, between 08:00 and 18:00. Each request sends 1,200 input tokens and gets 300 output tokens back. At lunch, from 12:00 to 14:00, traffic peaks at 5 requests per second. In a load test, one replica held 2 requests per second inside the team's latency target. At the peak, 5 / 2 = 2.5, so you need 3 replicas.
Scale to zero decides what a quiet night costs. With zero replicas, nothing runs and nothing is billed. In Kubernetes, scaling to zero works only on external or object metrics, such as a queue length. CPU or memory use can only be measured on running containers, and at zero none run. Kubernetes names jobs that need GPUs as a case where scaling to zero pays off, because idle GPUs are expensive.
The price of zero is the first request after it. That request waits while a replica starts, and how long it waits depends on the image and the model. We cover that in our article on serverless inference cold-start latency.
Deploy a Hugging Face model without building an image
For a model on Hugging Face, our dedicated deployments start from the model ID, so you build no image. Lyceum is a sovereign European AI inference provider for open-source models. A dedicated deployment runs your model on reserved GPU replicas and exposes it as an OpenAI-compatible Chat Completions endpoint. For dedicated endpoints, the data centre is agreed for your workload.
You reference the model by its Hugging Face model ID. For a gated model, you add a Hugging Face token when you create the deployment. Here is the example as a deploy command; the model ID is hypothetical:
lyceum infer deploy your-team/support-assistant-8b \
-g gpu.a100 \
--min-replicas 1 \
--max-replicas 3 \
--target-rps 2 \
--hf-token "$HF_TOKEN"The replica count scales between your minimum and maximum. Two targets decide when it scales: requests per second per replica, and the p95 latency, the time that 95% of requests beat. A stabilisation window stops the count from flapping up and down. By default it waits 300 seconds before it scales down. The full flag list is in our dedicated inference docs.
The work from the first section is done for you. We expose one OpenAI-compatible endpoint per deployment and balance load across its replicas. Each replica reports its status (pending, running, stopped or failed) and a health flag from periodic health checks. Incidents on our side appear on our status page.
You call the deployment with the OpenAI SDK, with the deployment ID as the model name:
from openai import OpenAI
client = OpenAI(
api_key="lk_...",
base_url="https://api.lyceum.technology/api/v2/external",
)
resp = client.chat.completions.create(
model="<deployment_id>",
messages=[{"role": "user", "content": "Hello"}],
)
print(resp.choices[0].message.content)Scale to zero is on for every deployment, with nothing to configure. After 1 hour without requests, the deployment pauses at zero replicas, and billing for idle replicas stops. The next request resumes it to your minimum. That first request is slower while the replicas start. For a walk-through of the whole path, see our guide to deploying a Hugging Face model as an API.
A day of traffic per token and per replica-hour
The two ways to serve the example are billed in different units, so put one day of each side by side. On a dedicated deployment, we bill for the time replicas run, not per request. On our per-token API, you pay for tokens, with no base fee.
If the model you need is one we already serve, the per-token side is simple arithmetic. DeepSeek V4.1 Flash costs $0.50 per 1M input tokens and $1.50 per 1M output tokens, at list prices on its model page on 6 November 2026:
input: 40,000 x 1,200 = 48M tokens x $0.50 per 1M = $24.00
output: 40,000 x 300 = 12M tokens x $1.50 per 1M = $18.00
day: $42.00 30 days: $1,260.00For your own fine-tune, the dedicated side counts replica-hours, one replica running for one hour. One replica runs from 08:00 to 18:00, then one idle hour before the pause. Two more run through the lunch peak, plus the 5-minute stabilisation window:
base replica: 10 h + 1 h idle before the pause = 11.00 replica-hours
peak replicas: 2 x (2 h + 5 min) = 4.17 replica-hours
day: 15.17 replica-hours (28.17 if the deployment never paused)| Way to serve | What you hand over | What it scales on | What you pay for | Example day |
|---|---|---|---|---|
| Per-token API | Nothing, we already serve the model | Our capacity, not yours | Input and output tokens | $42.00 |
| Dedicated deployment | A Hugging Face model ID | Requests per second and p95 latency per replica | Time the replicas run | 15.17 replica-hours |
The decision then fits in one line. Dedicated endpoints carry commitment pricing, agreed per contract, so the price per replica-hour comes from your quote:
dedicated costs less when: your price per replica-hour x 15.2 < $42.00 a dayA dedicated endpoint is priced for the capacity you reserve, so the busier you keep it, the less each token costs. Our comparison of pay per token and dedicated GPU inference works through that tradeoff in more detail.
The example favours neither side on purpose, and it leaves several things out. It ignores the cached-input price, which lowers the token bill when prompts repeat. It does not say how many GPUs one replica needs, which changes the replica-hour price. It also leaves out start-up time after a pause and the shape of your real traffic.
When you still need your own container
You still need your own image when the code around the model is yours. A custom server, extra processing inside the request path, or a model that does not live on Hugging Face all need a container. Then the four needs from the first section apply, and the health check and the scaling signal are yours to get right.
On our platform, a Docker image runs as a job today. Our docs send a one-off job that runs and exits to a regular Docker run. Production traffic with your own model goes to a dedicated deployment. Docker runs take a single image or a Docker Compose stack and stream their logs back. Private images pull with basic registry credentials or with AWS credentials for ECR.
So check in this order before you write a Dockerfile. First, is the model on our models page, so you can call it per token? Second, is it on Hugging Face, so a dedicated deployment can start from its ID? Only when both answers are no does the image become the thing you hand over.
If your traffic is steady enough for a dedicated endpoint, book a 30-minute call about dedicated endpoints. Bring three numbers: requests a day, peak requests per second, and the requests one replica holds in your load test.
Frequently asked questions
Can I deploy a gated Hugging Face model?
Yes. For a gated model, you pass your Hugging Face token when you create the deployment.
What happens to the first request after a quiet night?
After an hour without requests, the deployment is paused at zero replicas and billing for them stops. The next request resumes it to your minimum replica count, so that one request takes longer.
Can I test Lyceum for free before committing, for example with trial credits or a proof of concept?
For serverless inference, you pay as you go with no commitment, so you can start small and scale when you're ready. For dedicated inference, we run time-boxed proofs of concept with success criteria we agree on before you commit. The customer pays for these proofs of concept. They usually run for a few days, up to a week, depending on the workload.