Pick the fine-tuning method before you pick the GPU. Start with QLoRA or LoRA on one card, and move to a multi-GPU node only when an evaluation shows the adapter is not good enough.
You have a model and a few thousand examples of the task. Which GPU do you rent, and how many of them? The answer comes from a short memory calculation. We run it below for one example that carries through the article: a 27B model the size of Qwen3.8 27B. It is dense, meaning it uses all its weights for every token. The size is our choice for illustration, not a customer job.
How much GPU memory LLM fine-tuning needs
Fine-tuning memory is the training state the GPU holds while it updates the model. We assume BF16 weights (a 16-bit number format) and AdamW, the usual optimizer that turns gradients into weight updates. Then the state is 16 bytes per parameter before activations. The ZeRO paper (Rajbhandari et al., 2019) counts 2 + 2 + 12 bytes per parameter for Adam in mixed precision. Mixed precision means mixing 16-bit and 32-bit numbers.
- Weights: 2 bytes per parameter in BF16.
- Gradients: 2 bytes per parameter. A gradient is the correction the training step computes for each weight.
- Optimizer states: 12 bytes per parameter. AdamW keeps a 32-bit master copy of each weight plus two running averages.
- Activations: the intermediate results the model computes while reading a batch (the forward pass), kept until it computes the gradients (the backward pass). They grow with batch size and sequence length.
full fine-tuning memory = parameters x 16 bytes + activations
27B example = 27e9 x 16 bytes = 432 GB + activations
weights 54 GB
gradients 54 GB
optimizer 324 GBThree quarters of the 432 GB are optimizer states, not the model itself. No single GPU holds that, so full fine-tuning at 27B is a multi-GPU job. Activations come on top. Gradient checkpointing cuts them by recomputing parts of the forward pass, a trade of compute for memory that PyTorch documents.
How LoRA and QLoRA shrink the memory budget
LoRA (low-rank adaptation) freezes the base model and trains a small adapter of extra weights next to it. Low-rank means each adapter matrix is the product of two thin matrices, so it holds few numbers. Gradients and optimizer states then exist only for the adapter. Compared with fine-tuning GPT-3 175B with Adam, the LoRA paper reports 10,000 times fewer trainable parameters and GPU memory cut to about a third.
QLoRA goes one step further. It stores the frozen base model in 4-bit precision, about 0.5 bytes per parameter instead of 2. Storing weights at lower precision is called quantization. The QLoRA paper fine-tuned a 65B model on one 48 GB GPU while keeping the task performance of 16-bit fine-tuning. Hugging Face's PEFT library trains only a small number of extra parameters, with LoRA among its methods.
Unsloth, an open-source fine-tuning library, publishes minimum GPU memory per model size; some models need more. The full fine-tuning column is our arithmetic at 16 bytes per parameter, before activations.
| Model size | QLoRA (4-bit), minimum | LoRA (16-bit), minimum | Full fine-tuning, before activations |
|---|---|---|---|
| 7B | 5 GB | 19 GB | 112 GB |
| 14B | 8.5 GB | 33 GB | 224 GB |
| 27B (our example) | 22 GB | 64 GB | 432 GB |
| 70B | 41 GB | 164 GB | 1,120 GB |
For the 27B example, QLoRA needs about one twentieth of the memory of full fine-tuning. That ratio is why the method decides the hardware more than the model does. Our article on LoRA vs full fine-tuning covers the quality side of that choice.
Which GPU fits which fine-tuning job
The right GPU is the smallest one that holds your method's memory budget, with room left for activations. The table maps the 27B example onto the GPUs we offer as virtual machines. The H100 and A100 values are NVIDIA's SXM figures. VM sizes of 1, 2, 4 or 8 GPUs come from our machine types.
| GPU | Memory per GPU | Memory bandwidth | Link between GPUs | Smallest fit for the 27B example |
|---|---|---|---|---|
| L40S | 48 GB | 864 GB/s | PCIe Gen4, 64 GB/s | QLoRA on 1 GPU |
| A100 80GB | 80 GB | 2,039 GB/s | NVLink, 600 GB/s | LoRA on 1 GPU |
| H100 | 80 GB | 3.35 TB/s | NVLink, 900 GB/s | LoRA on 1 GPU; full on 8 GPUs |
| H200 | 141 GB | 4.8 TB/s | NVLink, 900 GB/s | Full on 4 GPUs |
| B200 | 192 GB | 8 TB/s | NVLink, 1.8 TB/s | Full on 4 GPUs |
The DGX B200 is NVIDIA's server with eight B200 GPUs. NVIDIA lists 64 TB/s of memory bandwidth and 14.4 TB/s of NVLink across its eight GPUs. The B200 row divides both by eight.
Full fine-tuning needs 432 GB, so it needs at least six 80 GB cards. VMs come in 1, 2, 4 or 8 GPUs, so in practice that means eight H100s with 640 GB. Four H200s hold 564 GB and four B200s hold 768 GB. Split across eight H100s, each card carries 54 GB of training state and keeps 26 GB for activations.
The faster card can be the cheaper run. MLPerf Training is an industry benchmark that times set training jobs. Blackwell is the generation of the B200, Hopper that of the H100 and H200. In the v4.1 round, NVIDIA reported 2.2 times Hopper's per-GPU performance for Blackwell on Llama 2 70B LoRA fine-tuning. That is a vendor result, so benchmark your own job. Compare what a finished run costs, not the hourly rate alone. For a 70B model the same logic applies at larger sizes; see which GPU to fine-tune a 70B model on.
When multi-GPU fine-tuning needs NVLink
A job that spans several GPUs has to split the training state across them, which is called sharding, and then exchange it on every step. DeepSpeed's ZeRO stages decide how much is split.
- ZeRO-1 splits the optimizer states across GPUs.
- ZeRO-2 also splits the gradients.
- ZeRO-3 also splits the weights and gathers each layer from the other GPUs during the forward and backward passes.
PyTorch's FSDP (Fully Sharded Data Parallel) shards model parameters across GPUs, and its documentation names ZeRO Stage 3 as an inspiration. ZeRO-3 is what lets eight H100s hold the 27B example. It also moves the most data between cards. NVLink, NVIDIA's direct link between GPUs, carries 900 GB/s per H100. The L40S has no NVLink. Its cards talk over PCIe Gen4, the standard slot connection in a server, at 64 GB/s, so they wait longer on every exchange.
That makes the L40S a single-card machine. In the 27B example, it fits QLoRA, which needs 22 GB of its 48 GB and no exchange at all. For a sharded run, rent cards joined by NVLink. Our guides on ZeRO-3 vs FSDP and multi-GPU training setup cover the configuration.
Fine-tuning on GPUs in European data centres
Lyceum is a sovereign European AI inference provider for open-source models. Our press facts state that the GPUs run in European data centres. For fine-tuning, you can run each step of the example on our hardware:
- Size the job first. The sizing tool on our compute page shows the memory a Hugging Face model needs and where it fits. For a PyTorch script, Pythia predicts training memory and runtime. It handles a single model on a single GPU, which covers the LoRA and QLoRA cases.
- Launch a virtual machine. GPU virtual machines come with root access and per-second billing, with B300, B200, H200, H100, A100 and L40S cards. Current rates are on our pricing page.
- Or submit the job. Serverless training runs your Docker container on the GPU you choose, with no machine to look after.
- Or let us train the adapter. Managed fine-tuning is in early access. You upload a few hundred examples, and we train a LoRA adapter on GLM-5.3 Flash or Qwen3.8 27B. The adapter is yours to download.
Pythia runs as the gpu-selection command of the Lyceum CLI. Before you rent anything, run it on your training script:
lyceum gpu-selection run train.py -- --epochs 3Arguments after `--` go to your script; here, three passes over the training data (epochs). The result names a recommended GPU and lists, for each GPU it tested, the predicted run time, cost, GPUs needed and memory use.
If the prediction fits one 80 GB card, launch a 1-GPU H100 or A100 VM and run LoRA there first. Test the adapter on a held-out set of your own examples against a quality bar you set first. Move to an 8-GPU node only when it misses that bar.
Summary: fine-tuning hardware and where Lyceum helps
Choose the method first, because it sets the memory. In the 27B example, full fine-tuning holds about 432 GB and needs an 8-GPU node. LoRA needs 64 GB on one 80 GB card. QLoRA needs 22 GB and fits one 48 GB L40S. Runs split across GPUs need cards joined by NVLink.
Lyceum covers the GPU choice, the billing and the data location in one place:
- One card or eight. Our virtual machines run 1, 2, 4 or 8 GPUs, from the L40S for QLoRA to H100, H200 and B200 nodes for full fine-tuning.
- No paying for idle hours. Per-second billing means a short LoRA trial costs only the minutes it runs.
- Data stays in Europe. Your training runs on GPUs in our European data centres.
- A sized plan before you pay. Pythia names the GPU your script needs, so you rent the right card the first time.
Sources
- Rajbhandari et al.: ZeRO, memory optimizations toward training trillion parameter models (arXiv PDF, 2019), section 3.1
- docs.pytorch.org
- arxiv.org
- Dettmers et al.: QLoRA, efficient finetuning of quantized LLMs (arXiv, 2023)
- Hugging Face: PEFT Parameter-Efficient Fine-Tuning Documentation
- Unsloth documentation: fine-tuning VRAM requirements (read 14 October 2026)
- NVIDIA: L40S GPU specifications (read 14 October 2026)
- NVIDIA: A100 Tensor Core GPU specifications (read 14 October 2026)
- NVIDIA: H100 Tensor Core GPU specifications (read 14 October 2026)
- NVIDIA: H200 Tensor Core GPU (read 3 August 2026)
- NVIDIA: DGX B200 (read 3 August 2026)
- NVIDIA Technical Blog: Blackwell doubles LLM training performance in MLPerf Training v4.1 (13 November 2024)
- DeepSpeed: Zero Redundancy Optimizer tutorial
- docs.pytorch.org