Lower your GPU bill: a guide to costs, billing and migration

Start with one run and its invoice. That gives you a specific charge to challenge and a way to check whether a change helped.

Why is our GPU bill so high, and which change should we make first?

The short answer

Reducing GPU cost starts with a useful unit of work: a completed training run, batch or inference task. The FinOps Foundation calls this unit economics: relating technology spending to the value it creates.

We recommend recording the job, hardware, allocation time, result and charge together. Add storage, transfers and commitments when they affect the decision. A run that fails its acceptance test belongs in the cost record too.

Lyceum is a sovereign European AI inference provider for open-source models. Our GPU virtual machines give you root access and per-second billing in European data centres. Those are useful controls when your team runs its own training code. They still require you to manage the machine’s lifetime.

Consider a hypothetical experiment on one GPU. The job finishes after 20 minutes, but the machine remains allocated for an hour.

Stopping after the verified output would remove 40 allocated minutes, or two-thirds of this example’s allocation. This is arithmetic, not a measured saving. It excludes startup, storage, transfers and any minimum commitment. It also assumes the same successful job finishes in the same time.

Step 1: Hourly Rates

Hourly rates help once you know which hardware and billing unit you are comparing. Read this if the advertised price looks lower than your invoice.

Record the GPU variant, GPUs per machine, memory, region, purchase term and quote date. Ask whether the listed rate covers one GPU or the whole machine. Then compare the cost of the same accepted job.

In our example, changing the hourly rate leaves the extra 40 minutes intact. First decide whether the expense comes from price, runtime or the machine staying alive afterwards.

Reading route:

• Renting an H100 in Europe
• B200 pricing and hardware
• Understanding a provider price change
• Reading an H100 node price

Use the dated comparisons to learn what to ask. Confirm current prices and terms before budgeting from them.

Step 2: Billing Models

Billing models decide which periods of your workload attract a charge. Read this if your usage changes between experiments, batches and steady production.

Our compute options include on-demand machines, spot machines, reserved capacity and serverless training. Spot capacity can be reclaimed. Serverless training runs your container on the GPU you choose; reserved capacity is priced per contract.

Ask what starts and stops the meter. A per-second rate does not itself stop a machine when the job finishes. A reservation also needs a different calculation from a short experiment: compare the whole commitment against work you expect to complete.

Reading route:

• Per-second billing comparison
• How per-second billing works
• Base fee versus usage-only
• Reserved versus on-demand capacity
• Training on spot instances

Before choosing spot capacity, test that the job can resume from saved progress. Include replayed work when comparing its bill.

Step 3: Idle Waste

Idle waste needs diagnosis because a running GPU can wait for several different reasons. Read this if jobs take longer than expected or machines stay allocated between jobs.

For our completed experiment, the first investigation is the shutdown path. For a job that is still running, inspect its execution timeline. PyTorch Profiler can record processor activity, tensor memory and operation timing.

Use that evidence to distinguish data preparation, computation and waiting. Change one suspected bottleneck, then rerun the acceptance test and compare elapsed time. A low activity reading alone does not tell you which change will help.

Reading route:

• Diagnosing low GPU utilisation
• Calculating idle GPU cost
• Spotting excess GPU capacity
• Cost per training run
• Reducing training costs

Before terminating one of our VMs, move the output you need to persistent storage. Our storage documentation explains that VM disks are wiped on termination, while your account’s storage bucket persists. Cost control should preserve the result you paid to produce.

Step 4: Rent vs Own

Renting versus owning is a decision about the whole operating period. Read this if you are considering a hardware purchase or a longer capacity contract.

Build two estimates for the same workload. For ownership, request hardware, power, cooling, space, maintenance and staffing costs. For rental, include the commitment, supporting services and work your team still operates. State how much completed work each estimate assumes.

Our example’s hour of allocation is too small a sample for this choice. Gather a representative history before extending a short experiment into a year-long forecast.

Reading route:

• GPU cluster ownership costs
• On-premise versus cloud breakeven
• Colocation versus cloud compute
• Dedicated GPU versus cloud instance
• Evaluating GPU investment returns
• Checking a GPU cluster quote

Keep the assumptions visible. Recalculate when expected demand, delivery dates or the acceptable turnaround time changes.

Step 5: Expired Credits

Expiring credits call for a forecast using the charges your team will actually pay. Read this if a grant or introductory balance has been hiding part of the invoice.

List each resource, its owner, current usage and the date its subsidy ends. Separate active production work from experiments that can be retired. Ask the provider which credits cover which charges, then model the first unsubsidised month.

Reading route:

• Planning for expiring cloud credits
• Responding when credits expire
• Building a predictable GPU bill

Use the run ledger to choose a small workload for any migration trial. A deadline is a reason to prioritise that test, not evidence that a replacement will cost less.

Step 6: Egress Fees

Egress means data leaving a service or network. Read this if your proposed move involves datasets, checkpoints or results crossing a charging boundary.

Draw the transfers before estimating them: source, destination, volume and frequency. Distinguish the initial migration from repeated traffic after the move. Request the applicable terms for each route, including any exception that depends on completing a migration.

Reading route:

• Comparing data-transfer charges
• Modelling data-transfer costs
• Evaluating egress-free offers

Keep these charges separate from the time calculation above. Removing 40 allocated minutes says nothing about the cost of moving a dataset.

Step 7: Hyperscaler Exit

Leaving a large cloud provider requires a workload test and a list of the services surrounding it. Read this if cheaper compute would mean replacing part of your current stack.

Start with the job’s code, dependencies, data access, secrets, logs and output location. Docker describes an image as an immutable package: once created, that image does not change.

Record the image version used in both environments. Then check the result and record the time and charges. Our serverless training accepts a plain Docker container, as shown on the compute page. A portable package helps the trial; it does not replace testing storage access or the surrounding workflow.

Reading route:

• Migrating to dedicated GPUs
• Replacing a managed training stack
• Moving ML workloads to Europe
• Evaluating alternative GPU infrastructure
• Comparing GPU purchasing options
• Training on independent infrastructure
• Switching to a European GPU cloud
• Assessing an alternative training cloud

Where to start, by situation

Choose the next investigation from the charge you can explain least well.

Change the billing choice when the measured workload supports it. To price the option you have identified, compare our current billing options.