Lyceum Magazine cover: the title Hardware recommendations for LLM fine-tuning by method, beside line art of a GPU card carrying one large outlined block and one small purple-topped block, suggesting a frozen model with a small trained adapter.

Pick the fine-tuning method before you pick the GPU. Start with QLoRA or LoRA on one card, and move to a multi-GPU node only when an evaluation shows the adapter is not good enough.

You have a model and a few thousand examples of the task. Which GPU do you rent, and how many of them? The answer comes from a short memory calculation. We run it below for one example that carries through the article: a 27B model the size of Qwen3.8 27B. It is dense, meaning it uses all its weights for every token. The size is our choice for illustration, not a customer job.

How much GPU memory LLM fine-tuning needs

Fine-tuning memory is the training state the GPU holds while it updates the model. We assume BF16 weights (a 16-bit number format) and AdamW, the usual optimizer that turns gradients into weight updates. Then the state is 16 bytes per parameter before activations. The ZeRO paper (Rajbhandari et al., 2019) counts 2 + 2 + 12 bytes per parameter for Adam in mixed precision. Mixed precision means mixing 16-bit and 32-bit numbers.

  • Weights: 2 bytes per parameter in BF16.
  • Gradients: 2 bytes per parameter. A gradient is the correction the training step computes for each weight.
  • Optimizer states: 12 bytes per parameter. AdamW keeps a 32-bit master copy of each weight plus two running averages.
  • Activations: the intermediate results the model computes while reading a batch (the forward pass), kept until it computes the gradients (the backward pass). They grow with batch size and sequence length.
full fine-tuning memory  = parameters x 16 bytes + activations
27B example              = 27e9 x 16 bytes = 432 GB + activations
  weights     54 GB
  gradients   54 GB
  optimizer  324 GB

Three quarters of the 432 GB are optimizer states, not the model itself. No single GPU holds that, so full fine-tuning at 27B is a multi-GPU job. Activations come on top. Gradient checkpointing cuts them by recomputing parts of the forward pass, a trade of compute for memory that PyTorch documents.

A stacked bar of the 432 GB that full fine-tuning of a hypothetical 27B model holds before activations: weights 54 GB, gradients 54 GB and AdamW optimizer states 324 GB, highlighted. Below, eight boxes for eight 80 GB H100s, each filled to 54 GB, showing that ZeRO-3 splits the state so each GPU keeps 26 GB for activations. Calculated example.
In this calculated example, 324 of the 432 GB that full fine-tuning of a 27B model holds are optimizer states, and ZeRO-3 on eight H100s leaves each GPU 26 GB for activations.

How LoRA and QLoRA shrink the memory budget

LoRA (low-rank adaptation) freezes the base model and trains a small adapter of extra weights next to it. Low-rank means each adapter matrix is the product of two thin matrices, so it holds few numbers. Gradients and optimizer states then exist only for the adapter. Compared with fine-tuning GPT-3 175B with Adam, the LoRA paper reports 10,000 times fewer trainable parameters and GPU memory cut to about a third.

QLoRA goes one step further. It stores the frozen base model in 4-bit precision, about 0.5 bytes per parameter instead of 2. Storing weights at lower precision is called quantization. The QLoRA paper fine-tuned a 65B model on one 48 GB GPU while keeping the task performance of 16-bit fine-tuning. Hugging Face's PEFT library trains only a small number of extra parameters, with LoRA among its methods.

Unsloth, an open-source fine-tuning library, publishes minimum GPU memory per model size; some models need more. The full fine-tuning column is our arithmetic at 16 bytes per parameter, before activations.

Model sizeQLoRA (4-bit), minimumLoRA (16-bit), minimumFull fine-tuning, before activations
7B5 GB19 GB112 GB
14B8.5 GB33 GB224 GB
27B (our example)22 GB64 GB432 GB
70B41 GB164 GB1,120 GB

For the 27B example, QLoRA needs about one twentieth of the memory of full fine-tuning. That ratio is why the method decides the hardware more than the model does. Our article on LoRA vs full fine-tuning covers the quality side of that choice.

Three horizontal bars on one gigabyte scale for a hypothetical 27-billion-parameter model. QLoRA needs at least 22 GB and fits one 48 GB L40S. LoRA needs at least 64 GB and fits one 80 GB H100 or A100. Full fine-tuning holds 432 GB before activations, highlighted, which needs eight 80 GB H100s with 640 GB. Calculated example.
For our hypothetical 27B model, QLoRA fits one 48 GB L40S and LoRA one 80 GB H100 or A100, while full fine-tuning needs eight H100s (Unsloth minimums, read 21 October 2026).

Which GPU fits which fine-tuning job

The right GPU is the smallest one that holds your method's memory budget, with room left for activations. The table maps the 27B example onto the GPUs we offer as virtual machines. The H100 and A100 values are NVIDIA's SXM figures. VM sizes of 1, 2, 4 or 8 GPUs come from our machine types.

GPUMemory per GPUMemory bandwidthLink between GPUsSmallest fit for the 27B example
L40S48 GB864 GB/sPCIe Gen4, 64 GB/sQLoRA on 1 GPU
A100 80GB80 GB2,039 GB/sNVLink, 600 GB/sLoRA on 1 GPU
H10080 GB3.35 TB/sNVLink, 900 GB/sLoRA on 1 GPU; full on 8 GPUs
H200141 GB4.8 TB/sNVLink, 900 GB/sFull on 4 GPUs
B200192 GB8 TB/sNVLink, 1.8 TB/sFull on 4 GPUs

The DGX B200 is NVIDIA's server with eight B200 GPUs. NVIDIA lists 64 TB/s of memory bandwidth and 14.4 TB/s of NVLink across its eight GPUs. The B200 row divides both by eight.

Full fine-tuning needs 432 GB, so it needs at least six 80 GB cards. VMs come in 1, 2, 4 or 8 GPUs, so in practice that means eight H100s with 640 GB. Four H200s hold 564 GB and four B200s hold 768 GB. Split across eight H100s, each card carries 54 GB of training state and keeps 26 GB for activations.

The faster card can be the cheaper run. MLPerf Training is an industry benchmark that times set training jobs. Blackwell is the generation of the B200, Hopper that of the H100 and H200. In the v4.1 round, NVIDIA reported 2.2 times Hopper's per-GPU performance for Blackwell on Llama 2 70B LoRA fine-tuning. That is a vendor result, so benchmark your own job. Compare what a finished run costs, not the hourly rate alone. For a 70B model the same logic applies at larger sizes; see which GPU to fine-tune a 70B model on.

A job that spans several GPUs has to split the training state across them, which is called sharding, and then exchange it on every step. DeepSpeed's ZeRO stages decide how much is split.

  • ZeRO-1 splits the optimizer states across GPUs.
  • ZeRO-2 also splits the gradients.
  • ZeRO-3 also splits the weights and gathers each layer from the other GPUs during the forward and backward passes.

PyTorch's FSDP (Fully Sharded Data Parallel) shards model parameters across GPUs, and its documentation names ZeRO Stage 3 as an inspiration. ZeRO-3 is what lets eight H100s hold the 27B example. It also moves the most data between cards. NVLink, NVIDIA's direct link between GPUs, carries 900 GB/s per H100. The L40S has no NVLink. Its cards talk over PCIe Gen4, the standard slot connection in a server, at 64 GB/s, so they wait longer on every exchange.

That makes the L40S a single-card machine. In the 27B example, it fits QLoRA, which needs 22 GB of its 48 GB and no exchange at all. For a sharded run, rent cards joined by NVLink. Our guides on ZeRO-3 vs FSDP and multi-GPU training setup cover the configuration.

Fine-tuning on GPUs in European data centres

Lyceum is a sovereign European AI inference provider for open-source models. Our press facts state that the GPUs run in European data centres. For fine-tuning, you can run each step of the example on our hardware:

  • Size the job first. The sizing tool on our compute page shows the memory a Hugging Face model needs and where it fits. For a PyTorch script, Pythia predicts training memory and runtime. It handles a single model on a single GPU, which covers the LoRA and QLoRA cases.
  • Launch a virtual machine. GPU virtual machines come with root access and per-second billing, with B300, B200, H200, H100, A100 and L40S cards. Current rates are on our pricing page.
  • Or submit the job. Serverless training runs your Docker container on the GPU you choose, with no machine to look after.
  • Or let us train the adapter. Managed fine-tuning is in early access. You upload a few hundred examples, and we train a LoRA adapter on GLM-5.3 Flash or Qwen3.8 27B. The adapter is yours to download.

Pythia runs as the gpu-selection command of the Lyceum CLI. Before you rent anything, run it on your training script:

lyceum gpu-selection run train.py -- --epochs 3

Arguments after `--` go to your script; here, three passes over the training data (epochs). The result names a recommended GPU and lists, for each GPU it tested, the predicted run time, cost, GPUs needed and memory use.

If the prediction fits one 80 GB card, launch a 1-GPU H100 or A100 VM and run LoRA there first. Test the adapter on a held-out set of your own examples against a quality bar you set first. Move to an 8-GPU node only when it misses that bar.

Summary: fine-tuning hardware and where Lyceum helps

Choose the method first, because it sets the memory. In the 27B example, full fine-tuning holds about 432 GB and needs an 8-GPU node. LoRA needs 64 GB on one 80 GB card. QLoRA needs 22 GB and fits one 48 GB L40S. Runs split across GPUs need cards joined by NVLink.

Lyceum covers the GPU choice, the billing and the data location in one place:

  • One card or eight. Our virtual machines run 1, 2, 4 or 8 GPUs, from the L40S for QLoRA to H100, H200 and B200 nodes for full fine-tuning.
  • No paying for idle hours. Per-second billing means a short LoRA trial costs only the minutes it runs.
  • Data stays in Europe. Your training runs on GPUs in our European data centres.
  • A sized plan before you pay. Pythia names the GPU your script needs, so you rent the right card the first time.

Sources

  1. Rajbhandari et al.: ZeRO, memory optimizations toward training trillion parameter models (arXiv PDF, 2019), section 3.1
  2. docs.pytorch.org
  3. arxiv.org
  4. Dettmers et al.: QLoRA, efficient finetuning of quantized LLMs (arXiv, 2023)
  5. Hugging Face: PEFT Parameter-Efficient Fine-Tuning Documentation
  6. Unsloth documentation: fine-tuning VRAM requirements (read 14 October 2026)
  7. NVIDIA: L40S GPU specifications (read 14 October 2026)
  8. NVIDIA: A100 Tensor Core GPU specifications (read 14 October 2026)
  9. NVIDIA: H100 Tensor Core GPU specifications (read 14 October 2026)
  10. NVIDIA: H200 Tensor Core GPU (read 3 August 2026)
  11. NVIDIA: DGX B200 (read 3 August 2026)
  12. NVIDIA Technical Blog: Blackwell doubles LLM training performance in MLPerf Training v4.1 (13 November 2024)
  13. DeepSpeed: Zero Redundancy Optimizer tutorial
  14. docs.pytorch.org