Which open-source LLM matches GPT-5 for long-context RAG?

Which open model can replace GPT-5 when you paste hundreds of pages into each prompt, and how do you check that it holds up? The question comes up when a workload moves off a closed API.

What long-context RAG asks of a model

RAG (retrieval-augmented generation) means your app finds relevant text and pastes it into the prompt before the question. Long-context RAG pastes a lot: whole contracts, manuals or ticket histories. Text is counted in tokens, small pieces of a word, and a model's context window is the most tokens it reads in one request.

A long prompt asks two things of a model. It has to find the lines that matter, often far apart. Then it has to combine them and ignore near-misses that look similar. A window that fits the text does not prove the model uses all of it. RULER, a 2024 research benchmark for long prompts, tested 17 such models. Only half of those that claimed 32,000 tokens or more kept satisfactory accuracy at 32,000.

GPT-5 has a 400,000-token context window. MiniMax M3, Kimi K3 and GLM-5.3 each list 1M tokens on our model pages.

Lyceum is a sovereign European AI inference provider for open-source models. Our serverless models, the shared models you call per token without running any servers, run in Paris and Finland. That matters when the documents you paste hold customer data. Prompts and outputs are processed, not stored, and never used for training. Open models now trail the best closed models by about four months on average. Epoch AI measured that gap on its Capabilities Index, a combined score across many benchmarks, on 29 May 2026.

Open models and GPT-5 on a long-document benchmark

AA-LCR v1.1 is Artificial Analysis's long-context reasoning test. It asks 100 hard questions, each over about 100,000 tokens of company reports, legal texts and similar documents. A model needs at least a 128,000-token window to take part. The table shows the scores as we read them on 23 October 2026. The setting column is the reasoning effort Artificial Analysis ran: max is a model's highest level, and high is a lower one. For GPT-5, high is the highest setting OpenAI offers. MiniMax M3 has no setting, because its reasoning is always on.

On this test, all four open models in the table score above GPT-5 (high). GPT-5.5 (high) sits among them. The limit is the length: every question stays near 100,000 tokens. If your RAG prompts run to 300,000 tokens, this table does not cover them. For how to read benchmark tables in general, see our guide to reading open model benchmarks.

The reasoning setting moves the score more than the model

On the same test, the reasoning setting changes a score more than the choice between the top open models. DeepSeek V4.1 Flash drops from 84.0% to 55.3% without reasoning. GLM-5.3 drops from 79.7% at max to 72.7% at low effort.

On our API, you set this with reasoning_effort, a request field that takes none, low, medium or high. The benchmark's max has no direct equivalent in that list, and we have not measured which value comes closest. The value none turns reasoning off on every model that supports switching. MiniMax M3 cannot turn it off. The reasoning text counts as output tokens, so set max_tokens high enough to hold it and the answer.

Some clients send none by default to save tokens. Check what yours sends before you compare models on long documents.

Our test: one two-step lookup at 33,000, 218,000 and 650,000 tokens

On 23 October 2026 we built a long synthetic supplier register and asked one question about it. The answer needed two lines far apart. One line said Halvorsen Logistics belongs to cost centre KX-48. Another said KX-48 has a cap of 41,300 euros. In the middle sat a decoy, a near-miss built to look right: Halvorsen Freight, cost centre KX-84, cap 14,300 euros. A model that answers 14,300 has followed the decoy. The same document counts as 216,000 to 226,000 tokens depending on the model, because each model splits text into tokens its own way. We call that length about 218,000. Reasoning on means we sent no reasoning_effort, so each model used its default. Off means we sent none.

Every model was right up to about 218,000 tokens with reasoning on. At about 650,000 tokens, MiniMax M3 returned the decoy's cap in its one run. With reasoning off, DeepSeek V4.1 Flash returned the decoy both times. Kimi K3 stayed right. GLM-5.3 reported no reasoning tokens, but wrote its working into the answer text. So its reasoning was not really off, and its result does not show how it does without reasoning.

Long prompts also take time. At about 218,000 tokens, a request took 14.2 to 14.6 seconds on DeepSeek V4.1 Flash. MiniMax M3 took 23.8 to 25.3 seconds, and Kimi K3 36.8 to 37.1. Current speeds for short requests are on our status page.

This is one synthetic lookup with one or two runs per setting. It shows how the four models behaved on our API on that day. It is not a benchmark, and we did not run GPT-5 in it.

What a 216,000-token prompt costs per question, and when retrieval wins

Every token you paste is billed as input. Take MiniMax M3 at its list prices read on 23 October 2026. Input costs $0.40 per 1M tokens, cached input $0.10 and output $2.00.

Prompt caching lowers the bill when requests repeat. It means that when consecutive requests start with the same text, our API reuses the work and bills those cached tokens at the lower rate. A request that finds its start in the cache is a cache hit. It is automatic and best-effort: a request can miss the cache when recent traffic has pushed the stored text out. In our test, we sent the same prompt to MiniMax M3 twice. It counted 216,399 tokens. The second response reported 216,320 cached tokens and came back in 7.1 seconds instead of 17.6.

Now count 1,000 questions over that one document set. The retrieval prompt of 8,000 tokens is hypothetical: the retrieved passages plus the question.

# Calculated example, MiniMax M3 list prices of 23 October 2026, per 1M tokens
full_prompt = 216_399 * 0.40 / 1e6                       # $0.0866 per question
cache_hit   = 216_320 * 0.10 / 1e6 + 79 * 0.40 / 1e6     # 79 = uncached remainder; $0.0217 per question
retrieval   = 8_000 * 0.40 / 1e6                         # $0.0032 per question
output      = 177 * 2.00 / 1e6                           # 177 output tokens incl. 174 reasoning; $0.0004
# per 1,000 questions: $86.56, $21.66, $3.20, plus $0.35 for output

Pasting the whole set costs $86.56 per 1,000 questions without the cache. With a cache hit on every call it costs $21.66, about a quarter. Retrieval at 8,000 tokens costs $3.20. Output adds about $0.35 in each case. That covers all 177 output tokens of our run, 174 of them reasoning, since reasoning is billed as output.

The example favours caching, because it assumes every call hits the cache. It does not show whether retrieval finds the right 8,000 tokens. That is the decision: retrieval wins when your search reliably returns the passages the answer needs. Long context wins when answers depend on lines your search would miss. Model size for the generation step is covered in our guide to open models for RAG.

How to test long-context quality on your own documents

Build the test from your own documents, so the length and the language match your app. Plant two facts far apart that only make sense together. Add one near-miss decoy, then ask a question that needs both facts. Run it at your real prompt length and at twice that length.

The script below runs that check against our OpenAI-compatible API. It sends reasoning_effort explicitly and prints the cached tokens from the usage record.

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["LYCEUM_API_KEY"],
                base_url="https://api.lyceum.technology/openai/v1")

docs = open("my_documents.txt").read()   # your text, with the two facts and one decoy planted
question = "What is the approved cap of the cost centre that Halvorsen Logistics is assigned to?"

for model in ["moonshotai/kimi-k3", "deepseek/deepseek-v4.1-flash", "z-ai/glm-5.3"]:
    for effort in ["high", "none"]:
        r = client.chat.completions.create(
            model=model, max_tokens=16000, timeout=600,
            reasoning_effort=effort,
            messages=[{"role": "user", "content": docs + "\n\n" + question}])
        d = r.usage.prompt_tokens_details
        print(model, effort, r.usage.prompt_tokens, d.cached_tokens if d else 0,
              r.choices[0].message.content.strip()[-80:])

Run each setting at least twice, since one run can be luck. Count right answers, decoy answers and request time per model. Then shortlist the cheapest model that stays right at your length with the reasoning setting you will run in production.

If a model holds up at twice your real length, you have headroom. If it fails at your real length, retrieve less or move to a model that passed. Create an API key in your dashboard and run the two-step test on your own documents.