Run open models without managing GPUs: choose the right endpoint

Start with a hosted model API when it meets your needs.

Consider a dedicated deployment when you need your own model, a different capacity setup or a better cost for measured traffic.

Step 01 / 05 · Pilot

Start with a model API that fits the feature

Serverless means paying for tokens processed, with no base fee and no GPUs to provision: it suits a pilot or changing demand.

  • Choose a model that supports your input, request size and required functions. Then test it on your own documents.
  • Set LYCEUM_MODEL to an available model ID from our library. The example request checks the connection; use real documents to test quality.
Before moving on: have a working request and evidence that the model can complete your task.
Exhibit 2Your application calls an OpenAI-compatible endpoint; the GPUs behind it are a shared servicerequest.flow
applicationclient.chat.completions.create
endpointapi.lyceum.technology/openai/v1
modelLYCEUM_MODEL
GPUsShared service: you do not provision or maintain them
Source Lyceum serverless inference, OpenAI-compatible API
Example: send a request with the OpenAI clientPython
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYCEUM_API_KEY"],
    base_url="https://api.lyceum.technology/openai/v1",
)
response = client.chat.completions.create(
    model=os.environ["LYCEUM_MODEL"],
    messages=[{
        "role": "user",
        "content": "Summarise this contract clause: ...",
    }],
)
print(response.choices[0].message.content)
Step 02 / 05 · Pilot

Measure the traffic and compare the costs

A break-even shows where two cost estimates are equal, not whether the quoted deployment can handle the traffic.

  • Record requests per day, input and output tokens, peak traffic and response times, over a period that includes normal variation rather than one unusually busy day.
  • A dedicated deployment runs your model on reserved GPUs. You pay for the time its replicas run; a replica is one copy of the model server.
  • Compare the full dedicated quote with the serverless cost of the same workload, with enough capacity for the speed and peak demand you need.
  • The example uses 3,000 input and 500 output tokens per request: $0.0064 per request at the recorded GLM-5.3 rates, before caching or extra reasoning output.
Before moving on: have measured traffic, a comparable quote and explicit assumptions behind the cost calculation.
Exhibit 3At 300 requests a day serverless costs $57.60 a month; it reaches the $4,000 quote at ~20,800Compare a 30-day month · $0.0064 per request
Requests a dayServerless per monthDedicated per month
300$57.60$4,000
3,000$576.00$4,000
~20,800 · break-even$4,000$4,000
30,000$5,760.00$4,000
Break-even: 625,000 requests per month
Source Assumes $0.0064 per request at GLM-5.3 prices recorded on 20 October 2026. The $4,000 starting quote is invented, not a Lyceum price. Excludes caching, additional reasoning, engineering and supporting services.
Step 03 / 05 · Operate

Decide what your application still owns

Managed serving runs the model; your application still decides what to send, what the model may do and how to handle the result.

  • Retrieval-augmented generation, often called RAG, means finding relevant documents and adding them to the prompt.
  • An agent adds a loop in which the model requests tools and responds to their results.
  • Both require application logic. Test the complete workflow, including missing or invalid function calls, before relying on it.
Before moving on: assign an owner to every part of the workflow outside model serving.
Exhibit 4Five responsibilities stay with your team when model serving is managed
ResponsibilityWhat your team must implement
DocumentsFind the relevant content and put it into the request.
ToolsValidate function arguments and control which actions are permitted.
QualityTest answers and detect important failures.
WorkflowManage conversation state and multi-step tasks.
RecoveryDecide when to retry, use a fallback or ask the user to try again.
Managed model serving: the model infrastructure
Source Lyceum
Step 04 / 05 · Operate

Handle failures and quiet periods

Failures need a wait limit and bounded retries; a quiet hour pauses a dedicated deployment and the next request waits.

  • Set a limit on how long a user can wait. Use bounded retries for temporary failures, with increasing waits and some random variation so clients do not retry together.
  • Stream responses when users benefit from seeing the answer as it is generated. Streaming does not remove the need for appropriate timeouts or error handling.
  • Log the status, timestamp and request trace ID. These let you connect an application failure to the provider’s records.
  • If that first-request delay is unacceptable, discuss the requirement before choosing the deployment. Check service levels and support commitments in the applicable contract.
Before moving on: test failures and the first request after a quiet period, not only normal traffic.
Exhibit 5A quiet hour pauses a dedicated deployment, even with a minimum replica settingdedicated.timeline
trafficRequests
quiet periodOne hour without requests
pauseReplica billing stopsA minimum replica setting does not disable this
next requestWaits while the deployment starts again
Source Lyceum dedicated deployments
Step 05 / 05 · Decide

Choose using the evidence from your pilot

Serverless stays while it meets your needs; dedicated has to earn the switch on the same acceptance tests.

  • Run the same acceptance tests on the proposed setup. Check peak demand, quiet periods, error handling and the work your team will operate.
  • Increase traffic gradually. Keep the previous endpoint available until the new setup performs as expected.
Before committing: confirm the endpoint meets the workload requirements and that your team can support the application around it.
Exhibit 6Three reasons to evaluate dedicated capacity: a model, control or quoted economics
serverless
Stay
  • While it meets your needs for model choice, quality, speed and cost
dedicated
Evaluate
  • A model outside the shared catalogue
  • More control over deployment
  • Economics supported by a quote and test
Source Lyceum

Your next workload starts here