Best open-weight model to replace GPT for classification and structured extraction

The best open-weight model to replace GPT depends less on the model name than on the shape of your request. An open-weight model is one whose trained weights are published, so any provider can run it. For classification, a cheap model with reasoning switched off is usually enough. For structured extraction, the model must return valid JSON every time, which matters more than raw ability.

Which open-weight model can replace GPT for classifying tickets and extracting fields into JSON? The question comes up once the GPT bill grows, or a customer asks where their documents are processed.

Classification and extraction are two different jobs

Classification and structured extraction ask different things of a model, so they deserve different first picks. Classification means choosing one label from a fixed list, such as "billing" or "bug report" for a support ticket. The answer is a few tokens long, and a token is a fragment of a word.

Structured extraction means copying named fields out of text into a fixed format. An invoice becomes a JSON object with a supplier, a date and a total. The format is usually written as a JSON Schema, a standard description of which fields an object has and what type each one is.

Since January 2026, the best open-weight models have trailed the best closed models by four months on average. Epoch AI, a research group, measured this on an index that combines many benchmarks (29 May 2026). Near-frontier quality no longer needs a US API.

Lyceum is a sovereign European AI inference provider for open-source models. Our serverless models run in Paris and Finland, and prompts and outputs are processed, not stored, and never used for training. You call them through one OpenAI-compatible API, so the GPT client you already have keeps working.

Which open-weight model to test first for each job

For classification, test Qwen3.8 Flash Next first; for extraction, test Qwen3.8 27B. The table lists the four models we would shortlist, with list prices read on our models page on 23 October 2026.

Reasoning is the working-out text that many current models write before their answer, returned separately from it. The models with an "-instant" ID are the same model at the same price, answering without that reasoning phase. For a label, that text only adds output tokens you pay for.

Qwen3.8 Flash Next and GLM-5.3 Flash share the lowest prices of the four. Its reasoning can be switched off, which GLM-5.3 Flash does not allow cleanly. That makes it the first model to try for labels.

Qwen3.8 27B costs more per output token but follows a forced function call and lets you switch reasoning off. For extraction, that combination matters more than price. MiniMax M3 is the pick for long documents, since its context window holds 1M tokens, but you always pay for its reasoning.

Forcing structured output with a tool call

To get JSON from these models, we recommend a forced tool call. A tool call (or function call) is a structured reply in which the model fills in the arguments of a function you defined. You define one function whose parameters are your JSON Schema. Then you set `tool_choice` to that function, which tells the model it must call it.

import json
from pathlib import Path

from openai import OpenAI

client = OpenAI(base_url="https://api.lyceum.technology/openai/v1", api_key="lk_...")
invoice_text = Path("invoice.txt").read_text()

invoice_schema = {
    "type": "object",
    "properties": {
        "supplier": {"type": "string"},
        "invoice_date": {"type": "string", "description": "YYYY-MM-DD"},
        "total_eur": {"type": "number"},
    },
    "required": ["supplier", "invoice_date", "total_eur"],
}

resp = client.chat.completions.create(
    model="qwen/qwen3.8-27b",
    reasoning_effort="none",
    messages=[{"role": "user", "content": invoice_text}],
    tools=[{"type": "function", "function": {"name": "save_invoice", "parameters": invoice_schema}}],
    tool_choice={"type": "function", "function": {"name": "save_invoice"}},
)
calls = resp.choices[0].message.tool_calls
if calls:
    print(json.loads(calls[0].function.arguments))

Not every model enforces `tool_choice`. In our checks on 17 September 2026, DeepSeek V4.1 Flash and GLM-5.2 ignored a forced call and could answer in plain text instead. Kimi K3, GLM-5.3 Flash, MiniMax M3 and Qwen3.8 27B honoured it. Our serverless docs keep that list current. Whatever the model, check that `tool_calls` is not empty, and validate the arguments against your schema in code before you store them.

Keep each schema narrow. ExtractBench, a 2026 benchmark of document-to-JSON extraction, found that performance fell sharply as schemas grew wider. On a 369-field financial schema, every frontier model it tested produced 0% valid output. If a document needs 60 fields, ask for them in several calls of a dozen each.

How you ask counts as much as which model you ask. LLMStructBench compared 22 models and five prompting strategies on extraction into JSON. It found that the prompting strategy mattered more than model size. Keep your GPT prompt as the starting point, then test one or two variants on each candidate. For JSON reliability in more depth, see our guide to open models with reliable function calling and JSON output.

What reasoning and caching do to the monthly bill

Two request settings move the bill as much as the choice between these models, or more: reasoning and prompt caching. Prompt caching works on requests that start with identical text, such as a fixed system prompt. The repeated part is billed at a lower cached rate. That shared opening text is called the prefix.

Take a hypothetical classifier, our assumption and not a customer measurement. It labels 1,000,000 support tickets a month into 12 categories. Each request sends a fixed 600-token system prompt with the label definitions, plus a 200-token ticket. It returns a 10-token label.

monthly cost = (cached tokens x cached price + new input tokens x input price + output tokens x output price) / 1,000,000

With Qwen3.8 Flash Next, reasoning off and no cache hits, the 800M input tokens cost $160 and the 10M output tokens cost $5. That is $165 a month. If every 600-token prefix hits the cache, the cached part costs $30, the tickets $40 and the labels $5: $75 a month.

Reasoning tokens count as output tokens. In the third row, 300 reasoning tokens per ticket raise output from 10M to 310M tokens, which costs $155 instead of $5. The label stays the same, and the bill triples.

The example favours Flash Next: short labels and a long shared prompt suit a model with cheap input and caching. Caching is best effort, so a month with every prefix cached is the lowest the bill can go, not a forecast. Put the fixed system prompt first and the ticket last, so the shared text stays identical. The example also says nothing about accuracy. Only your own test can show whether Flash Next labels your tickets as well as GPT.

Testing an open model against your GPT baseline

An open model can replace GPT when it matches your GPT baseline on your own inputs. Your GPT baseline is the set of results your current GPT setup produces on those inputs. A two-step test compares the open model with it. Treat the steps below as our proposed test plan, not a published result.

First, collect a few hundred real inputs and have a person check the correct label or fields for each. Run them through your GPT model and through each candidate with the same prompt. Point your existing OpenAI client at our base URL, and change only the API key and the model ID.

Second, compare four numbers per model:

1. The share of correct labels, per label, so a weak rare class does not hide in the average.
2. The share of fields that match exactly, per field.
3. The share of responses with a valid tool call that passes your schema check.
4. The cost per 1,000 requests, read from your usage data rather than from list prices.

Public benchmarks only point you to candidates. Our guide on how to read open model benchmarks explains why. If you extract from scans or images, start with our article on the best open model API for OCR and document extraction instead.

When an open model still misses your baseline on a narrow task, fine-tuning is the next step. It suits narrow tasks with a clear right answer, where your team already has good examples. It is in early access.

If the candidate matches your GPT baseline on all four numbers, move the classification or extraction workload to it. Start with the per-token prices for each model and run your test set through our API this week.