What's new

What should you know before you send a 2.4-trillion-parameter model your production traffic?

Qwen released the open weights of Qwen3.8 2.4T A95B on 12 August 2026. We serve it on Serverless Inference, our hosted pay-per-token API, under the model ID `qwen/qwen3.8-2.4t-a95b`. It takes text in and gives text out. It does not read images today, and image requests fail with a 400 error.

The model always reasons before it answers. It first writes its working notes, called the reasoning trace, and then the answer. You cannot switch the reasoning off: the request is accepted, but the setting is ignored (see our API documentation).

On 20 October 2026, our status checks showed a median time to first token (the wait before the first word arrives) of 301 ms. Output speed was 102 tokens per second. Both are medians over the last seven days for short requests. Reasoning tokens count toward the time to first token and toward the output speed alike. Live numbers are on our status page.

Specs and context window

The context window is the amount of text, counted in tokens, that the model can read in one request. On Lyceum it is 256K tokens. Qwen's card lists 262,144 tokens natively, extensible up to 1,010,000 tokens.

SpecValue
Total parameters2.4 trillion
Active parameters per token95 billion
Experts512
Layers92 (23 full attention, 69 Gated DeltaNet)
Input and outputText only
LicenceQwen3.8-Max License, as named on our model page

A mixture-of-experts (MoE) model sends each token to a few of its many small expert networks. Here each token uses 10 of the 512 experts plus one shared expert, which is why 95 billion of 2.4 trillion parameters are active.

Gated DeltaNet layers keep a fixed-size summary instead of a KV cache (the stored record of every token read), so only the 23 full-attention layers keep a full cache.

Hardware requirements

The weights decide the hardware, because every parameter must sit in GPU memory even though only 95 billion work per token. The vLLM recipe for Qwen3.8 2.4T A95B lists how many GPUs each precision needs. Precision is the number of bits stored per weight: BF16 and FP8 are 16-bit and 8-bit formats, and NVFP4 is NVIDIA's 4-bit format. Weights are in TiB (tebibytes, about 1.1 TB each).

PrecisionWeights (TiB)NVIDIA B300 (268 GB each)NVIDIA H200 (141 GB each)
BF16 (16-bit)4.4524 GPUs48 GPUs
FP8 (8-bit)2.2716 GPUs32 GPUs
NVFP4 (4-bit)1.328 GPUs16 GPUs

The counts are the recipe's, not our measurement. We recommend testing the model through the API before you plan hardware.

Horizontal bars of GPU counts the vLLM recipe lists for Qwen3.8 2.4T A95B weights. BF16: 24 NVIDIA B300 or 48 H200. FP8: 16 B300 or 32 H200, highlighted. NVFP4: 8 B300 or 16 H200.
The vLLM recipe asks for 16 B300 or 32 H200 GPUs at 8-bit, and more at 16-bit; these counts are the recipe's, not our measurement.

Benchmarks, as the vendor reports them

Qwen's model card lists Terminal Bench 2.1 (a command-line agent test) at 86.6 and SWE-bench Pro (a test of fixing real software issues) at 67.7. Both appear under the name Qwen3.8-Max, which is not the open checkpoint we serve. The card's own results note says figures under that name "may not be identical to the raw open checkpoint". We have not measured these figures ourselves, so read them as the vendor's reference, not as a result for the model we serve.

Pricing

You pay per token, with no base fee. Prices are in US dollars and were read from the model page on 20 October 2026.

Token typePrice per 1M tokens
Input$2.50
Cached input$0.63
Output$6.00

Prompt caching reuses the work done on text a model has already read. It is automatic: when requests share an identical start, the repeated part is billed at the cached rate. Caching is best effort, so check `usage.prompt_tokens_details.cached_tokens` in the response. On a miss the field is absent, and you pay the normal input rate. Our API documentation describes both.

Take a coding-agent turn that sends the same 40,000 tokens as the last turn, adds 2,000 new tokens and gets 1,000 tokens back.

CaseInput costOutput costTotal
Cache hit$0.0252 (40,000 cached) + $0.0050 (2,000 new)$0.0060$0.0362
Cache miss$0.1050 (42,000 uncached)$0.0060$0.1110
Miss against hit3.1 times

This is a calculated example, not a measurement. It favours caching, because it assumes the whole 40,000-token start is found in the cache. It also assumes the 1,000 output tokens include the reasoning trace.

How to call it

Two horizontal stacked bars of one coding-agent turn on Qwen3.8 2.4T A95B. A cache hit costs 0.0362 dollars: 0.0252 cached input, 0.0050 new input, 0.0060 output. A cache miss costs 0.1110 dollars: 0.1050 input and 0.0060 output. Calculated example.
In our worked example, a cache hit costs $0.0362 per turn and a miss $0.1110, with the same 1,000 output tokens in both.

You call Qwen3.8 2.4T A95B with the OpenAI SDK, pointed at our base URL, and the model ID as `model`. We run sovereign European AI inference for open-source models, billed per token. This model runs on EU-hosted infrastructure. We don't keep your prompts or outputs, and we never use them for training.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYCEUM_API_KEY"],
    base_url="https://api.lyceum.technology/openai/v1",
)

stream = client.chat.completions.create(
    model="qwen/qwen3.8-2.4t-a95b",
    messages=[{"role": "user", "content": "Find the bug in this function: ..."}],
    max_tokens=8192,
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

The `max_tokens` value covers the reasoning trace and the answer together. A low value can end the request before the answer starts. To use the model from a coding tool, follow our Claude Code guide.

Where it fits in the Lyceum catalogue

The 2.4T model is the large option in the Qwen3.8 family. The two smaller models read images and cost less per token. All three have a 256K context window.

ModelInputPrice per 1M tokens, input / outputMedian time to first tokenOutput speed
Qwen3.8 2.4T A95BText$2.50 / $6.00301 ms102 tokens/s
Qwen3.8 27BText and images$0.40 / $2.40312 ms41 tokens/s
Qwen3.8 Flash NextText and images$0.20 / $0.50560 ms68 tokens/s

Speeds are medians over seven days for short requests, measured on 20 October 2026. Output is priced higher than on the 27B model ($6.00 against $2.40 per million tokens). Use the 2.4T model for long agent runs and hard coding tasks where the 27B answers fall short. Our guide to open models for agentic coding compares more options.

Run it on Serverless Inference: open the model page, create an API key and send the request above.

FAQ

Sources

  1. developer.nvidia.com
  2. huggingface.co
  3. recipes.vllm.ai