The number that decides your route is not a minimum order. It is whether your job fits on one 8-GPU machine. Up to 8 GPUs, you rent a virtual machine. Past 8, you need several machines, and that means a reserved cluster.
So teams often ask us: do we have to take 8 or 16 GPUs, or can we rent just one or two?
Why GPU capacity seems to come only in big blocks
GPU capacity looks like it comes in big blocks because the hardware does. A data-centre GPU server holds its GPUs on one board. NVIDIA sells its standard server board, called HGX, with eight GPUs on it. Providers rent that machine whole, or slice it.
Capacity lists add to the impression. Our own capacity table lists blocks of 128 GPUs and more, on terms of a year or longer.
That does not mean a small team is shut out. It means you should know which of three sizes your job really needs before you ask anyone for capacity.
Size the job by GPU memory before you pick a GPU count
GPU memory decides the count, so size it first. A model's weights must fit in the memory of the GPUs that run it. EleutherAI's Transformer Math guide gives the rules of thumb we use below.
In 16-bit precision, each parameter (one learned number in the model) takes 2 bytes. Serving needs a little more for working memory, so EleutherAI suggests about 1.2 times the weights. Full fine-tuning, which updates every parameter, needs far more. For each parameter, training keeps the weight itself (2 bytes) and its gradient (2 bytes), the direction of the next update. It also keeps the optimizer's working copies (12 bytes). The optimizer is the algorithm that applies each update, and EleutherAI's numbers are for AdamW, the common choice.
serve (16-bit): memory ≈ parameters × 2 bytes × 1.2
full fine-tune: memory ≈ parameters × (2 + 2 + 12) bytes, before activationsHere is a hypothetical example we carry through the article: a team with a 70-billion-parameter model. Serving it in 16-bit needs 70 × 2 = 140 GB of weights, and 140 × 1.2 = 168 GB in total. Fully fine-tuning it needs 70 × 16 = 1,120 GB, before the memory for activations (the intermediate results of each training step).
You can check your own model with the sizing tool on our compute page. It shows the memory a Hugging Face model needs and how many requests it serves.
One GPU, one 8-GPU machine or several machines
Every job lands in one of three sizes: one GPU, one machine with several GPUs, or several machines. The docs of vLLM, an open-source engine for serving models, draw the same lines. If a model fits on one GPU, you do not need to split it. If it fits on one node (one machine), you spread each layer over that machine's GPUs, so they work on it together. This is called tensor parallelism. Only a model too large for one machine also needs pipeline parallelism. Then each machine holds a group of layers and passes its results to the next.
The memory per GPU decides where our example lands. On our VMs an H100 has 80 GB, an H200 141 GB and a B200 192 GB. Our VMs come in sizes of one, two, four or eight GPUs, so 168 GB on H100 takes four, not three.
| Job (hypothetical 70B model) | GPU | GPUs needed | GPU memory | Route |
|---|---|---|---|---|
| Serve in 16-bit (168 GB) | H100 | 4 | 320 GB | One virtual machine |
| Serve in 16-bit (168 GB) | H200 | 2 | 282 GB | One virtual machine |
| Serve in 16-bit (168 GB) | B200 | 1 | 192 GB | One virtual machine |
| Full fine-tune (1,120 GB) | B200 | 8 | 1,536 GB | One virtual machine |
| Full fine-tune (1,120 GB) | H100 | 16 | 1,280 GB | Reserved cluster (two machines) |
Serving fits on one machine with every GPU. Fine-tuning is where the 8-GPU line matters: on B200 it still fits on one machine, on H100 it needs two. For the setup itself, see our guide to tensor parallelism across GPUs, and for more sizing cases, how much VRAM a 70B model needs.
The example leaves out activation memory, long contexts, cost and throughput (how many requests per second the GPUs serve). Treat it as the first cut, then size your real workload.
Rent 1, 2, 4 or 8 GPUs on a virtual machine
For jobs that fit on one machine, rent a GPU virtual machine. Lyceum is a sovereign European AI inference provider for open-source models: our GPUs run in European data centres. Your data is never used for training. For teams that run their own models, we rent everything from a single GPU to a whole cluster.
Each GPU type on our VMs launches with 1, 2, 4 or 8 GPUs, subject to availability and account limits. You get root access, and you pay per second with no subscription. Our billing docs add that there is no monthly minimum.
Spot VMs use the same GPUs at about 50% less. We can reclaim them at any time, so they suit checkpointed jobs, batch evaluations and reruns. A checkpoint is a saved state your job can restart from. Read more on spot GPUs for training.
In our example, serving fits on one B200 or two H200, so the team starts on a single VM today.
Reserve a cluster when the job spans several machines
A job that outgrows one machine runs across several of them, on a reserved cluster. Up to 8 GPUs on one machine, a VM is enough, unless you want a fixed pool of capacity held for you for months. Our clusters run H100, H200 or B200 machines on a 400 Gb/s InfiniBand network. InfiniBand is a fast link between machines, which training across several machines needs. Clusters are reserved per contract, for 1 week to 1 year or more, and there is no self-service launch.
Reserved capacity is priced per contract. The reservation planner on our compute page offers 8, 16, 32 up to 1,000+ GPUs in total, or a number you enter. You can reserve them for 1 to 36 months. The 16 H100 in our example are a typical two-machine request.
Our compute page also shows a Current capacity table with each block's earliest start and minimum term. When we read it on 6 November 2026, it listed blocks of 128 to 1,024 GPUs, with minimum terms of 12 to 36 months. Those were the blocks listed that day, not a minimum order: the planner also takes requests for 8 or 16 GPUs. If nothing listed fits, you book a call and we look for capacity. You can also add a backup GPU in the planner, a second GPU type you would accept. Before that call, check what a GPU cluster quote should contain and how reserved and on-demand capacity differ.
What to do when the GPU you need is out of stock
When a GPU is out of stock, ask to be told when it comes back. On the Launch Instance page, an out-of-stock profile shows a Notify when available button. We email you, and optionally call a webhook, as soon as the capacity is live.
Set the GPU count you really need. A reminder for 8 GPUs fires only when 8 are free at once, not when one is. If your job can run on fewer, set the smaller count, or leave the count out to hear about any free GPUs. For other options, see what to do when on-demand GPUs sell out.
The decision comes down to one check. If your job fits on one machine, launch a VM today with the GPU count it needs. If it needs more than 8, bring your memory estimate and start date to a reservation call.
Book a 30-minute call about reserved GPUs
Frequently asked questions
Can I rent 2 or 4 GPUs instead of a full machine?
Yes. You can launch a virtual machine with two or four GPUs, as long as that count is in stock and within your account limits.
Is a spot VM a good way to start small?
For jobs that save checkpoints, yes. Spot costs about half, but we can reclaim the machine at any time, so plan for restarts.
What does a GPU reservation request ask for?
The planner asks for the GPU type, the number of GPUs, the start date, how long you need them and the region. Your answers fill in the booking form for the call.