Choose an open model: compare fit, quality and cost

Start with the task and the input it needs.

Shortlist models that can handle both, compare their full token costs and test them on your own examples before choosing.

Step 01 / 05 · Shortlist

Match the model to the task

Write down what the feature needs first, and keep different jobs separate: retrieval and answering are different roles.

  • Write down what the feature needs before comparing model names. For the document assistant: reading contract text, finding the relevant clause and answering accurately.
  • Check each candidate for the input types it accepts, such as text or images, the amount of text it can handle in one request, function calling if the application needs it, and its intended tasks and deployment options.
  • Use our models page to identify candidates. Model descriptions help you shortlist; your own tests decide whether a model fits.
Before moving on: have a shortlist that supports the input and behaviour your feature requires.
Exhibit 2An embedding model can find the clause; generating the answer is a different role
inputcontract
embedding modelclausetext → numbers → similar passages
generating the answeranswerclause → answer: a different role
Source Illustration, Lyceum
Step 02 / 05 · Shortlist

Check that your longest requests fit

An average request can fit while an important customer’s request fails.

  • A context window is the amount of information a model can handle in one request, measured in tokens: the units used to process and bill text.
  • Check the provider’s limits for input and output. Leave room for the answer, conversation history, tool definitions and any reasoning that counts towards the limit.
  • In the example, a request has 6,000 input tokens and a 400-token answer: a 6,400-token starting estimate, before any additional output.
  • If a full document does not fit, decide whether the application should split it, retrieve relevant passages or use another model, and test that choice for answer quality too.
Before moving on: confirm that representative long requests fit with enough room for a complete answer.
Exhibit 3Size the context window for your longest real request, not the average onecontext window
✓ average request
input · 6,000400room
6,000 in + 400 outroom left before the limit
✕ longest real request
documentconversation historytool definitionsanswer
over the limit: the request fails
Source Illustration, not to scale. Example request: 6,000 input and 400 output tokens.
Step 03 / 05 · Compare

Calculate cost using your token mix

The same token mix costs $13.00 to $255.00 a day by model, and even that assumes every repeated prefix is cached.

  • Compare input, cached-input and output prices separately. Cached input is repeated prompt text the service can reuse at a lower rate.
  • Keep stable instructions and repeated document text at the beginning of the prompt. Caching requires a matching start and is best effort, so estimate both with and without cache hits.
  • The example: 20,000 requests a day, each with 6,000 input and 400 output tokens; 5,000 of the input tokens are repeated instructions and contract text. Measure your real cached share before budgeting.
  • Include reasoning tokens in output where they are billed. A model with longer reasoning or answers may cost more than a comparison using identical output counts suggests.
Before moving on: have a cost range for your traffic and clear assumptions about cache hits and output length.
Exhibit 4On the same 20,000 requests a day, GLM-5.3 Flash costs least and Kimi K3 mostCost per day · every repeated prefix hits the cache · 5,000 cached input tokens per request
GLM-5.3 Flash
$13.00
MiniMax M3
$34.00
Qwen3.8 27B
$37.20
GLM-5.3
$89.20
Kimi K3
$255.00
Source Hypothetical traffic. Prices recorded on 26 October 2026. The cached case is an optimistic scenario; first requests and cache misses pay the regular input rate. Quality, speed and extra reasoning output are not included.
Step 04 / 05 · Compare

Test quality and speed on your own prompts

Public benchmarks measure other tasks; a fixed test set of your own real requests makes the final choice.

  • Build a fixed test set from real requests, including difficult cases. Define what counts as a pass before running the candidates.
  • For the document assistant, a pass might require the answer to identify the correct clause and avoid making claims the contract does not support.
  • Run two or three suitable candidates. For each, record how many answers pass, how long answers take to start and finish, total token cost including failed attempts, and function-call or formatting failures.
Before moving on: identify the candidate that meets your quality and speed requirements at an acceptable cost.
Exhibit 5Every candidate runs the same test set, and failed attempts count in the total cost
test setReal requestsincl. difficult cases; pass defined first
candidatesA, B, Csame set for each
compare oncost per successful tasktotal cost ÷ successful tasks
Source Illustration, Lyceum
Step 05 / 05 · Release

Record the exact model and review it before release

A specific model ID makes the choice explicit, not permanent: check retirement notices and rerun tests before releases.

  • Record the model ID, settings, licence and processing region used in your tests, and keep that configuration with your evaluation results.
  • Read the licence for the exact version you use. Check the processing region on its model page before sending data with location requirements.
  • Start with a small rollout and keep the previous configuration available until the new model meets your requirements in production.
Before launch: have a recorded configuration, licence and region checks, and a way to switch back.
Exhibit 6With a routing alias the provider picks the model, and the model behind it may change
specific model ID
model ID → model
  • Makes the choice explicit
  • Does not guarantee identical output or indefinite availability
routing alias
alias → provider → underlying model
  • Can reduce selection work
  • The model behind the alias may change: decide whether that suits your workflow
Source Illustration, Lyceum

Your next workload starts here