Memory is rarely the limit for Whisper. Speed is, and it decides what each audio hour costs. So the useful question is not which GPU fits the model, but how many audio hours one GPU transcribes per hour. What does that make a transcription API cost?
How much GPU memory Whisper Large v3 needs
Whisper Large v3 needs about 10 GB of GPU memory (VRAM) in OpenAI's reference code, and under 5 GB in an optimised engine. OpenAI lists the large model at 1,550 million parameters and about 10 GB of required VRAM. Large v3 has the same architecture as the earlier large models. It reads 128 Mel frequency bins instead of 80: finer frequency bands in the audio picture the model reads. It also adds a language token, the tag that tells the model which language to write, for Cantonese.
The faster-whisper project measured memory and time for 13 minutes of audio. Faster-whisper runs Whisper on CTranslate2, an inference engine with its own GPU kernels, the small programs that do the maths on the GPU. The test used large-v2 on an RTX 3070 Ti, a consumer graphics card with 8 GB. Large v3 has the same architecture and size, so expect similar memory and speed.
| Implementation | Precision | Time for 13 minutes | VRAM |
|---|---|---|---|
| openai/whisper | 16-bit | 2 min 23 s | 4,708 MB |
| faster-whisper | 16-bit | 1 min 3 s | 4,525 MB |
| faster-whisper | 8-bit | 59 s | 2,926 MB |
| faster-whisper, batch of 8 | 8-bit | 16 s | 4,500 MB |
Precision is how many bits store each weight. Quantization, storing weights in fewer bits, here means 8 bits instead of 16. Here it cut memory from 4,525 MB to 2,926 MB and barely changed the time. A batch is a group of audio segments the GPU processes together. Batching is what made the run 3.7 times faster than the single-stream 8-bit run.
On an NVIDIA L40S, a data-centre GPU with 48 GB, those 4.5 GB are 9.4% of the memory. So you do not pick the GPU for the model's size. You pick it for how fast it works through your audio.
Choose a serving engine for throughput
The engine, the software that loads the model and runs it, sets the throughput: audio processed per second of GPU time. This guide compares three engines.
faster-whisper is the default for batch jobs. It decodes audio itself with PyAV, a Python library that bundles FFmpeg's decoders, so FFmpeg does not need to be installed. Its batched pipeline turns on voice activity detection (VAD) by default. VAD is a small model that finds the parts of a file with speech, so Whisper skips the silence. In the code, int8_float16 means 8-bit weights with 16-bit calculations.
from faster_whisper import WhisperModel, BatchedInferencePipeline
model = WhisperModel("large-v3", device="cuda", compute_type="int8_float16")
batched = BatchedInferencePipeline(model=model)
segments, info = batched.transcribe("call-0001.mp3", batch_size=8)
print(" ".join(s.text for s in segments))
vLLM is the choice when your app expects an OpenAI-style audio API. vLLM is an open-source serving engine. It lists Whisper among its transcription models and serves the /v1/audio/transcriptions endpoint. Your code keeps the OpenAI SDK and changes only the base URL.
Transformers with Flash Attention 2 is the third route. Attention is the step where the model weighs which parts of the input matter, and Flash Attention 2 computes it faster. The insanely-fast-whisper project used it on an A100 80 GB, an NVIDIA data-centre GPU, with a batch of 24. It transcribed 150 minutes of audio in 1 minute 38 seconds. That is a different GPU and engine from the table above, so treat it as a second data point.
If you only need English or a faster model, look at turbo. OpenAI describes it as an optimised large-v3 with 809 million parameters, about 6 GB of VRAM and about 8 times the speed. It is not trained for translation.
From GPU hours to a cost per audio hour
The cost per audio hour is the GPU's price divided by how many audio hours it transcribes per hour. The real-time factor (RTF) is processing time divided by audio length. Its inverse, the speed-up, is the number you plan with.
Take a hypothetical backlog of 10,000 hours of recorded support calls. Use the 8-bit, batch-of-8 result from the table as the speed. That run took 16 seconds for 780 seconds of audio.
RTF = 16 s / 780 s = 0.0205
speed-up = 780 s of audio / 16 s of compute = 48.75
so one GPU hour transcribes 48.75 audio hours
GPU hours = 10,000 / 48.75 = 205.1
cost = GPU hours × the GPU's hourly rate on our compute page
On an L40S at the on-demand rate on our GPU compute page, read 30 October 2026, 205.1 GPU hours cost $408.21. That is about $0.04 per audio hour. The example assumes the L40S is at least as fast as the 8 GB test card, which nobody has measured. Measure your own speed-up before you budget.
A GPU also costs money while it waits for files to download or decode. Divide by the share of time it is busy:
busy 70%: 205.1 / 0.70 = 293.0 GPU hours
cost: 293.0 × L40S price = $583.15, about $0.06 per audio hour
More GPUs shorten the wall-clock time but not the GPU hours. At full speed, 205.1 GPU hours take 51.3 hours on 4 GPUs and 25.6 hours on 8.
| Setup | GPU hours | Cost (prices read 30 October 2026) |
|---|---|---|
| L40S on demand, 48.75× speed-up, always busy | 205.1 | $408.21 |
| L40S on demand, 48.75× speed-up, 70% busy | 293.0 | $583.15 |
| A100 spot, 91.8× speed-up (other engine and GPU) | 108.9 | $129.58 |
The A100 row uses the insanely-fast-whisper result, so it mixes another engine and another GPU. Its speed-up is 9,000 seconds of audio in 98 seconds, so 91.8×. The cost uses today's A100 spot rate on the same compute page. Spot machines cost about 50% less but can be reclaimed at any time. That suits a backlog where each file is written out when done, so a restart loses little. The table is a calculated example: it does not show your accuracy, your silence share or your file sizes.
Prepare the audio so the GPU stays busy
Throughput falls when the GPU waits for CPU work, so keep decoding and filtering off the GPU's path. Whisper hears 30 seconds at a time, its receptive field. Longer files are cut into windows in one of two ways.
The model card calls them sequential and chunked. Sequential slides a window through the file in order. Chunked cuts the file into 30-second pieces that overlap a little and transcribes them in parallel. For batches of long files, the card says sequential is about as fast as chunked. It is also up to 0.5% better in word error rate, the share of words transcribed wrongly. Use chunked when one long file must come back quickly.
Three habits keep the GPU fed:
- Decode and run VAD in separate CPU processes, and pass ready segments to the GPU process. Faster-whisper's VAD removes silence longer than 2 seconds by default.
- Write each transcript as soon as its file is done. A reclaimed spot machine then loses one file, not the run.
- Time one hour of your own audio end to end before you size anything. That measured speed-up replaces 48.75 in every line above.
Run Whisper on our GPUs
Run a live transcription API on a GPU virtual machine. Run a backlog as a Docker run that stops when it is done. We are Lyceum, a sovereign European AI inference provider for open-source models, built and run in Europe. The GPUs run in European data centres. On our serverless API, prompts and outputs are processed, not stored, and never used for training. On a GPU virtual machine, you decide what is stored.
Our GPU virtual machines give you root access and are billed per second while they run. You choose 1, 2, 4 or 8 GPUs per machine. On a machine, start vLLM with Whisper and call it like the OpenAI API:
vllm serve openai/whisper-large-v3
from openai import OpenAI
client = OpenAI(base_url="http://YOUR_VM_ADDRESS:8000/v1", api_key="unused")
with open("call-0001.mp3", "rb") as f:
result = client.audio.transcriptions.create(model="openai/whisper-large-v3", file=f)
print(result.text)
For a backlog, package your faster-whisper script in a container image, push it to your registry and submit it with our CLI. A Docker run mounts your storage at /mnt/s3 by default. Its timeout defaults to 300 seconds, so set -t for a long job. The command uses the A100, the default machine type for runs:
docker build -t your-team/whisper-batch . && docker push your-team/whisper-batch
lyceum docker run your-team/whisper-batch -m gpu.a100 -t 86400 \
-c "python transcribe.py /mnt/s3/audio /mnt/s3/transcripts"
Our dedicated endpoints, a model on GPUs reserved for you, serve a Hugging Face model through an OpenAI-compatible Chat Completions endpoint. So they are not the route for a transcription API. For how the endpoint types compare, see our guide to choosing an endpoint. If transcription is one stage of a longer media pipeline, our AI dubbing cost guide prices the other stages.
The decision comes down to how your audio arrives. A steady stream needs a machine that stays up. A backlog needs a job that runs and stops. Start an L40S or A100 machine on our compute page, time one hour of your own audio, and size the rest from that number.
Frequently asked questions
Can I call Whisper through your serverless API?
Our serverless catalogue on the models page lists text models and embedding models, which turn text into vectors. Whisper is not among them as of 30 October 2026. Run Whisper on a GPU virtual machine or as a Docker run instead.
Is Whisper Large v3 turbo good enough?
Turbo is about 8 times faster and needs about 6 GB of VRAM, according to OpenAI. It is not trained for translation, so keep Large v3 when you translate speech into English.