A Claude prompt is plain text, so you can send its wording to an open-weight model unchanged. An open-weight model is one whose trained weights are published, so any provider can run it. What breaks is the request around the words: answers you start for the model (prefills), reasoning settings and the output format.
Budget output tokens for reasoning before anything else. In our own test, the model's hidden working took most of each answer. A cap set for Claude's short answers can leave nothing to read.
How do you migrate prompts written for Claude to an open-weight model without losing quality? This guide sorts your Claude habits into keep and replace, then shows the test that decides the switch.
Why a working Claude prompt behaves differently on an open model
A Claude prompt was tuned against one model and one API. Moving it changes both.
The first change is the chat format. A chat template is the model's own rule for turning a list of messages into the token sequence it was trained on. Control tokens are special markers for where a message starts and ends. Hugging Face's documentation notes that models can use different formats and control tokens, even when they come from the same base model. Your system prompt (the instructions sent before the user's message) arrives wrapped in different markers.
The second change is reasoning. Many open models write a reasoning trace before the answer: their working, billed as output tokens. Our serverless inference documentation shows it switched on by default on most models.
The move is worth the work now. Since January 2026, the best open models have trailed the best closed ones by four months on average, according to Epoch AI (29 May 2026).
What carries over: XML tags, roles and examples
Keep the content of your prompt: the instructions, the examples and the structure. Anthropic recommends XML tags such as <instructions> and <context> because they separate instructions, context, examples and inputs. It also recommends setting a role in the system prompt.
To an open model, those tags are ordinary text inside the prompt. Your prompt was tuned on Claude, not on the new model. Keep the tags, and let your test set decide whether they still work.
Keep the order too. Prompt caching (reusing the work already done on repeated prompt text) is automatic on our API. When requests start with the same text, that earlier work is reused at a lower input price. Put the stable part, such as the system prompt and the examples, at the front, and the changing input at the end.
What to replace: prefills, reasoning settings and output tricks
Prefill. A prefill is a partial assistant reply you write for the model to continue, such as a lone { to force JSON. Anthropic itself stopped supporting it: from the Claude 4.6 models on, a prefilled last assistant turn returns a 400 error. Anthropic's replacements work on open models too: a format rule in the instructions, and "Respond directly without preamble" in the system prompt.
Token limits. A reasoning model spends output tokens before it writes the answer. If max_tokens (the cap on output tokens per request) is sized for Claude's short answers, the budget can run out mid-thought. The answer then comes back empty or cut off.
Output tricks. A format that Claude followed may need a stricter instruction elsewhere. The table lists the faults from our own test and our documentation, with the fix for each.
| What you see | Cause | Fix |
|---|---|---|
| Empty answer | Reasoning used the whole token budget | Raise max_tokens, or switch reasoning off where the model allows it |
| JSON inside a code fence | The model formats code by habit | Say "no code fence" in the prompt, or strip fences in your parser |
Reasoning text ending in </think> in the answer | Reasoning switched off on GLM-5.3 | Leave reasoning on for GLM-5.3 |
No tool call despite tool_choice: "required" | GLM-5.3 does not enforce it today | Check message.tool_calls and fall back when it is empty |
More on tools: open models with reliable function calling and JSON output.
Chat templates and reasoning switches on our API
On our API, you send a list of messages and the model's chat template is applied for you. There is no legacy text /completions endpoint, so you never write raw template markers yourself.
Reasoning is the setting you do control. reasoning_effort: "none" turns it off on every model that supports switching. Clients that prefer the chat template's own switch can send chat_template_kwargs instead. Its key differs by model family. Other reasoning_effort values are passed to the model as sent, and our documentation says they are not reliably graded. So treat reasoning as on or off.
| Model | Reasoning by default | How to change it |
|---|---|---|
| GLM-5.3 | On | Leave it on; switching it off moves the reasoning into the answer |
| DeepSeek V4.1 Flash | On | reasoning_effort: "none" |
| Qwen3.8 27B | On | reasoning_effort: "none", or the model ID qwen/qwen3.8-27b-instant |
The trace comes back in message.reasoning_content on most models and in message.reasoning on some, including GLM-5.3. Read both fields.
Here is what reasoning costs on a real prompt. On 6 November 2026 our content team sent the same extraction prompt to three models. It asked for JSON with invoice_number, total_eur and due_date from a German invoice, and said "Reply with JSON only". It was one user message, with no prefill. All three models returned the correct values.
| Model | Input tokens | Output tokens | Of which reasoning | Cost per call | Per 10,000 documents |
|---|---|---|---|---|---|
| Kimi K3.1 Pro | 155 | 93 | 51 (55%) | $0.001240 | $12.40 |
| GLM-5.3 | 79 | 258 | 213 (83%) | $0.001246 | $12.46 |
| DeepSeek V4.1 Flash | 96 | 77 | 46 (60%) | $0.000164 | $1.64 |
Token counts are measured, temperature 0. Costs are calculated at our list prices of 9 November 2026. The volume of 10,000 documents a month is hypothetical.
cost per call = input tokens x input price / 1,000,000
+ output tokens x output price / 1,000,000For GLM-5.3, that is 79 x $1.40 / 1M plus 258 x $4.40 / 1M, which gives $0.000111 + $0.001135 = $0.001246.
Three things stand out. The same prompt counted as 79 to 155 input tokens, because each model splits text into tokens its own way. Reasoning was most of the output, and so most of the cost. And GLM-5.3 wrapped its JSON in a markdown fence although the prompt said JSON only.
One prompt is a smoke test, not a benchmark. It favours the models: the prompt is short and has one clear right answer. It shows no Claude baseline, no quality on your documents and no latency under load. On a longer coding prompt in the same test, DeepSeek V4.1 Flash hit our 16,000-token cap while still reasoning and returned no answer.
Migrate one prompt step by step
- Freeze a test set. Collect 20 to 50 real inputs with known right answers. The number is a suggestion.
- Remove the prefill. Move the format rule and "respond directly without preamble" into the system prompt.
- Keep tags, examples and order. Stable text first, so caching can reuse it.
- Set reasoning on purpose. Raise
max_tokensto cover the reasoning, or switch it off where the model supports it. - Change the endpoint. Point your client at our base URL with a Lyceum API key and the model name.
- Run and parse. Send every test case and parse each answer with your production code.
We are Lyceum, a sovereign European AI inference provider for open-source models: built and run in Europe. Our GPUs run in European data centres. Prompts and outputs are processed, not stored, and never used for training. Our API is OpenAI-compatible, so the endpoint step is a base URL and a model name.
from openai import OpenAI
client = OpenAI(api_key="lk_...", base_url="https://api.lyceum.technology/openai/v1")
resp = client.chat.completions.create(
model="z-ai/glm-5.3",
max_tokens=16000,
messages=[
{"role": "system", "content": "<instructions>Extract the invoice fields. Reply with JSON only, no code fence.</instructions>"},
{"role": "user", "content": "<input>Rechnung Nr. 2026-1147 ...</input>"},
],
)
msg = resp.choices[0].message
reasoning = getattr(msg, "reasoning_content", None) or getattr(msg, "reasoning", None)
print(msg.content)Other API-level differences, such as parameters that behave differently, are in what breaks when switching OpenAI-compatible APIs.
Test the migrated prompt before you switch traffic
Score what your code consumes: valid JSON, the right fields, the right tool call. Fluent prose your parser rejects is a failure.
Track two faults separately from answer quality: empty answers and fenced JSON. They come from settings and format, not from the model's ability, and the table above fixes both. If your answers come back empty on a reasoning model, check the token budget first.
Then decide. Switch when the migrated prompt meets your pass rate on the test set at a cost per call you accept. If only format faults remain, fix the prompt or the parser before you try another model. For a fuller method, see how to evaluate an open model against your current one.
Book a call to benchmark an open model against your Claude prompt on your own workload.
Frequently asked questions
Can I use open models in Claude Code?
Yes. Claude Code connects to our Anthropic-compatible endpoint. Claude Code picks a model by tier name, such as Fable or Opus. By default we map Fable to Kimi K3.1 Pro and Opus to GLM-5.3, and your Claude Code settings can change the mapping.
Do I still need cache breakpoints?
No. On our API, caching is automatic for requests that start with the same text. Keep the stable part of the prompt first.
Can I force a tool call on every model?
Not on every model. GLM-5.3 does not enforce a required tool call today, so check whether a tool call came back and fall back when it did not.