Understand what runs underneath

Technical guides, model comparisons and practical notes on running AI workloads.

LatestSwitch models safely · Open equivalents

Best Open-Weight Alternative to Claude Opus, and What Switching Saves

Kimi K3.1 Pro is the open-weight model closest to Claude Opus 5.5: it scores 46 against Opus's 48 at default, at half the token price. If you run Opus at xhigh or max effort, switching saves 39% to 63% per task. We host it in the EU and do not store your prompts.

Justus Amen

Lower inference cost · Commercial terms

EU LLM Inference Providers with Free PoCs or Trial Credits

A free credit helps only if it pays for a test on your own prompts. A proof of concept (PoC) is a short, time-boxed trial on GPUs reserved for you, with goals agreed first. With us, new accounts get a $20 credit after a card check, and our credits never expire.

Justus Amen

Meet EU requirements · Data residency

European models or European hosting: what compliance needs

Privacy law cares where your prompts go and who handles them, not where the model was built. The model's maker has to document the model, wherever it is based. We run the model you pick on GPUs reserved for you, in a European data centre you agree with us.

Magnus Grünewald

Meet EU requirements · Data residency

Germany-only LLM inference: serverless or dedicated?

If your data must stay in Germany, a shared pay-per-token API will not do. You need GPUs reserved for you, in a German location your contract names. Our shared API runs in Paris and Finland, and we agree dedicated locations per contract.

Magnus Grünewald

Run models without infra work · Custom deployments

Hand Over a Docker Image, Get an Autoscaling API

You need your own Docker image only when your serving code is custom. For a Hugging Face model, our dedicated deployments need only its ID and give you an API that scales. If we already serve the model, you call our per-token API with no deployment at all.

Maximilian Niroomand

Meet EU requirements · Data residency

Is the UK covered by EU inference? UK data residency for LLM APIs

No: an EU-hosted LLM API runs in EU countries, and the UK is not one of them. UK companies may still send personal data to one, because UK law approves that transfer. We run our serverless models in Paris and Finland, which suits contracts that allow EU locations.

Magnus Grünewald

Secure GPU capacity · Reservations

Is there a minimum GPU order? Renting 1 vs 8 vs 16 GPUs

There is no minimum order for a GPU virtual machine with us. You can rent one GPU or a full machine of 8, and pay per second. A job that needs more than 8 GPUs spans several machines and runs on a reserved cluster.

Justus Amen

Switch models safely · Migration plan

Migrating Prompts Written for Claude to an Open-Weight Model

Keep your Claude prompt text as it is. Change three things in the request: remove any reply you pre-start for the model (a prefill), allow enough tokens for the model's reasoning, and check the answer format in your code. To try it on our API, change the base URL.

Maximilian Niroomand

Meet EU requirements · Provider vetting

Neocloud funding risk: how to vet your inference provider

You can't see a private GPU cloud's finances. Check instead what you would lose if it stopped, and how fast you could move. Our open models work with any OpenAI client, so your code can move with you.

Justus Amen

Run models without infra work · GPU efficiency

Quantization on dedicated inference: NVFP4 vs FP8 vs full precision

Storing a model's numbers in 8 bits instead of 16 halves its GPU memory and usually keeps its answers. Going down to 4 bits saves more memory, costs a little accuracy and needs NVIDIA's Blackwell GPUs. On our dedicated endpoints, we agree this setting with you before launch.

Caspar Lehmkühler

Switch models safely · Open equivalents

Which Open-Weight Models Match Claude Sonnet Quality?

Kimi K3.1 Pro is the open-weight model that scores closest to Claude Sonnet 5.5 on an independent quality test. It scores 46, between Sonnet's 42 at its high thinking setting and 55 at its highest. You can run it on our API in Paris and Finland.

Maximilian Niroomand

Switch models safely · Evaluation

Why large LLM evals fail on serverless inference and how to run them

Most failed requests in a large eval come from your eval tool giving up too early, not from the API. Many tools wait only 30 seconds and then resend the request, while a long answer can take minutes. Our serverless API gives each request up to 300 seconds and tells you why a request failed.

Caspar Lehmkühler

Switch models safely · Evaluation

How to evaluate open-source LLMs against your proprietary model before switching

Send the same sample of your own production prompts to an open model and to your current model. Score both on quality, speed, cost and errors, and decide with limits you wrote down before the test. You can run the open model on our API in Paris and Finland, and we don't keep your prompts.

Maximilian Niroomand

Switch models safely · Open equivalents

Open-weight replacements for Claude Haiku and GPT-5.4 mini

DeepSeek V4.1 Flash and GLM-5.3 Flash come nearest Claude Haiku 5.5 on an independent quality test. No current test scores GPT-5 nano or GPT-5.4 mini, so try the same two on your own tasks. You can call both on our API, on GPUs in Paris and Finland.

Maximilian Niroomand

Lower inference cost · Token & cache savings

How to reduce token usage without hurting output quality

Send each request only the earlier messages and search results its answer needs. Ask for shorter answers, and turn off the model's thinking step where a test allows it. Keep the start of every prompt identical: on our API, that repeated part is billed at a lower rate automatically.

Justus Amen

Lower inference cost · Price comparisons

What open models actually cost per million tokens

On Lyceum's API, input to the open models below costs $0.20 to $3.00 per million tokens. Output costs $0.30 to $15.00 per million, so your mix of the two sets the price. We bill per token, and the models run on our GPUs in Paris and Finland.

Justus Amen

Switch models safely · Evaluation

LLM-as-a-Judge: comparing model outputs when you switch providers

With LLM-as-a-judge, a third model reads both answers to a prompt and picks the better one. Ask it twice with the answers swapped, and count a win only when both verdicts agree. You can run the candidate and the judge on our API, and we never store your prompts.

Maximilian Niroomand

Meet EU requirements · GDPR & contracts

Which LLM APIs don't train on your data

Check every provider's written policy for two separate promises: no training on your data, and no storage of it. A provider can keep training off by default and still store your prompts and responses. Lyceum does neither: we don't keep your prompts or outputs, and we never use them for training.

Magnus Grünewald

Meet EU requirements · Transfer risk

Which LLM API providers are European-owned and outside the CLOUD Act?

A provider is outside the US CLOUD Act only if no company handling your prompts answers to US courts. A European owner is a good start, but the parent company and the GPU host count too. We run our serverless models in Paris and Finland, and our Lyceum gateway never logs your prompts.

Magnus Grünewald

Meet EU requirements · Sector rules

LLM inference for healthcare data in Europe: where patient data may go

In Europe, patient data may go to an LLM API only under a health-data exception and a processing contract. German health providers also need EU processing, a German office and a recent audit against the federal cloud standard C5. We run our serverless models in Paris and Finland, and our gateway never logs the patient text.

Magnus Grünewald

Run models without infra work

Run open models without managing GPUs: choose the right endpoint

Start with a hosted model API when it meets your needs. Consider a dedicated deployment when you need your own model, a different capacity setup or a better cost for measured traffic.

Caspar Lehmkühler · 8 min read

Meet EU requirements · GDPR & contracts

AI gateway fallback routing without breaking EU data rules

Can your gateway send overflow to a second provider when your main one hits its limits, and still keep processing in the EU? Yes. The backup needs the same DPA, your written approval and EU processing as your main provider. Only capacity errors (429, 503) should send traffic there.

Magnus Grünewald

Model library · Launches

Day-zero model launches: how fast new open models go live on inference APIs

A new open model can run on a European API only once its files are public. The launch post often comes days earlier. Kimi K3.1 Pro was announced on 24 October 2026 and went live on our API on 5 November. You can call it with us in European data centres, and we don't keep your prompts.

Maximilian Niroomand

Model library · Changelog

DeepSeek V4 Pro retires on 19 November: how to move to V4.1 Flash

We retire DeepSeek V4 Pro on our API on 19 November 2026, and we will not move your requests to another model for you. From 5 November we move its capacity, the GPUs that answer its requests, to DeepSeek V4.1 Flash, so V4 Pro may answer more slowly. Change the model name in your code to deepseek/deepseek-v4.1-flash before then.

Maximilian Niroomand

Meet EU requirements · Provider vetting

Pinning Model IDs: How to Make Sure Serverless Inference Runs the Model You Requested

If you list the approved models in your AI gateway, can you be sure the approved model is the one that answers? Not with the list alone. Some model names are aliases, and with a routing alias the provider picks the model for you. So check twice: let only exact approved model IDs through, and read the model name in every response, which says which model actually answered.

Magnus Grünewald

Scale without limits · Latency & throughput

Priority inference tiers vs dedicated endpoints: getting guaranteed latency

A shared AI API sends every customer's requests to the same pool of GPUs. When that pool is busy, requests wait, and paying for a better place in the line does not make the line move faster. Reserved GPUs do, because their capacity is planned around your traffic alone.

Maximilian Niroomand

Keep your coding tools · Tool setup

Run Codex on open models with Lyceum's Responses API

Can you keep using Codex and switch it to open models? Yes, you can run Codex on open models hosted in Europe. Add Lyceum as a provider in Codex's config file and set your API key. You don't need to learn a new tool. Long sessions stay cheap because Codex resends the conversation on every turn, and we bill the repeated part at a lower price.

Maximilian Niroomand

Meet EU requirements · Data residency

Region pinning for LLM APIs: controlling where requests are processed

Region pinning means a provider processes your requests only in data centres in the places you choose. You set it by the service you call and by the data processing agreement (DPA), the contract covering your data. Check where the model runs, and also where the provider's staff and subcontractors can reach your data.

Magnus Grünewald · 8 min read

Meet EU requirements · Provider vetting

Sovereign LLM API for German enterprises: what to check

A sovereign LLM API for a German company does not have to run in Germany. Under the GDPR, the EU data protection law, a location in the European Economic Area (EEA) counts the same as Germany. What your approvers check is where requests run, who outside the EEA gets data, and the processing agreement, called AVV in German.

Magnus Grünewald · 7 min read

Model library

Choose an open model: compare fit, quality and cost

Start with the task and the input it needs. Shortlist models that can handle both, compare their full token costs and test them on your own examples before choosing.

Caspar Lehmkühler · 5 min read

Switch models safely · Task fit

Best open-weight model to replace GPT for classification and structured extraction

To sort text into a fixed set of labels, start with Qwen3.8 Flash Next and switch its reasoning off. To copy fields from text into a fixed JSON format, start with Qwen3.8 27B, which follows a forced function call. Test both on your own data against your GPT results before you switch.

Caspar Lehmkühler · 6 min read

Meet EU requirements · Data residency

EU-sovereign inference: model-by-model data residency

Data residency means the place where your data is processed. For an LLM API, that place depends on the model each request reaches, not on the company that sells access. Before you send personal data, check the stated region of each model your code calls, including any model an alias picks for you.

Magnus Grünewald · 5 min read

Switch models safely · Migration plan

Routing between open-source and proprietary models in one application

Put one small function in front of your model calls. It reads each request and picks a model by your rule, such as task type or personal data in the text. Open models are served behind APIs that speak the same format as OpenAI's, so both sides use the same client code.

Maximilian Niroomand · 7 min read

Switch models safely · Migration plan

Migrate an LLM workload from a US inference provider to the EU

Move one workload at a time, starting with a list of your app's model requests and the personal data they carry. Pick an open model we serve, then change three settings in your OpenAI client: the address, the key and the model name. Update your data processing paperwork and test on real traffic before you switch over.

Maximilian Niroomand · 8 min read

Switch models safely · Evaluation

What a two-week PoC with a new inference provider should include

A two-week proof of concept (PoC) sends a sample of your real requests to a new provider. It checks answer quality, your slowest response times, failure handling and cost per answered request against limits set on day one. Spend the first week on integration and quality, and the second on load and failures.

Justus Amen · 8 min read

Switch models safely · Task fit

Which open-source LLM matches GPT-5 for long-context RAG?

On Artificial Analysis's long-document reasoning test, GPT-5 (high) scores 78.2%. Four open models we serve score Kimi K3 88.7%, DeepSeek V4.1 Flash 84.0%, MiniMax M3 83.0% and GLM-5.3 79.7%. The test gives each model about 100,000 tokens of documents per question, so test your shortlist on your own documents at your real length.

Caspar Lehmkühler · 6 min read

Model library · Launches

DeepSeek V4.1 Flash 1017: what changed and how to switch

DeepSeek V4.1 Flash 1017 is DeepSeek's update of V4.1 Flash, at the same price on our API: $0.50 input and $1.50 output per million tokens. To use it, set the model value to deepseek/deepseek-v4.1-flash-1017 on the same endpoint and API key. We support the old version until midnight CET on 27 October, then route its requests to the new model.

Maximilian Niroomand · 6 min read

Meet EU requirements · Sector rules

EU-hosted LLM APIs for legal documents: what to verify before use

Client contracts can go to an EU-hosted LLM API only after four checks. The checks cover what data you send, which law protects it, what the provider stores, and how the model treats your documents. German lawyers must also choose the provider carefully and sign a secrecy contract, as section 43e of the Federal Lawyers' Act (BRAO) requires.

Magnus Grünewald · 7 min read

Meet EU requirements · GDPR & contracts

Before an LLM PoC: DPA, NDA and the customer-data checklist

Before you send customer data to an LLM API, you need a data processing agreement (DPA) with the provider. A DPA is the contract that limits what the provider may do with the personal data you send it. Add a non-disclosure agreement (NDA), a contract that keeps business information secret, only where you need that on top.

Magnus Grünewald · 6 min read

Meet EU requirements · Certifications

C5 requirements for LLM APIs in German public-sector projects

C5 is a catalogue of minimum cloud security requirements from Germany's Federal Office for Information Security (BSI). An auditor checks a provider against it and writes a C5 report. Your tender is met only if that report covers the exact LLM API service, region and period you will use.

Magnus Grünewald · 8 min read

Meet EU requirements · Sector rules

EU-Hosted LLM APIs for banking and insurance

Ask every provider for four written answers: where it processes data, who its subprocessors are, how you can audit it and how you can leave. A subprocessor is a company that runs part of the service for the provider. DORA, the EU Digital Operational Resilience Act, requires your bank or insurer customer to put such terms into its contract with you.

Magnus Grünewald · 7 min read

Switch models safely

Switching to an open model: five checks before you go live

Changing your API settings is only the first step. Before switching, check that the new model gives useful answers, works with your application and meets your requirements for speed and cost.

Maximilian Niroomand · 7 min read

Keep your coding tools

Keep your coding tools when you switch to open models

You can connect Claude Code, Codex, Cursor and GitHub Copilot to our models, but the supported features differ. Check the features your team needs, configure one tool and test a real coding task before rolling it out.

Maximilian Niroomand · 6 min read

Scale without limits · Latency & throughput

Good average latency, slow outliers: fixing P95 on an LLM API

Averages hide slow outliers. P95 is the wait that 95 out of 100 requests stay under, and the slowest few usually queue behind a burst at peak. Measure P95 in your own client, then size your endpoint for the busiest load that still meets your target, not for the average.

Maximilian Niroomand · 8 min read

Avoid vendor lock-in · Open model trust

Open model licences compared: what you can actually ship

Whether you can ship an open model in a product depends on the conditions in its licence file. Some licences ask only for a copyright notice, while others need a separate licence at a revenue limit or for a kind of product. Check the exact model before you ship.

Magnus Grünewald · 8 min read

Meet EU requirements · Provider vetting

Sovereign European AI inference: what the label must prove

The European Commission's Cloud Sovereignty Framework gives approvers a usable structure for the word sovereign. We turn the parts an inference buyer can verify into five checks: contract and law, request path, retention, exit and evidence. Each check ends in a document you can read or a gap you can name.

Magnus Grünewald · 7 min read

Lower inference cost · Unit economics

What AI code review costs per day, and how teams cut it

What does your AI code review cost per day, and how do you make it cheaper? A review bot runs again on every push to a pull request, and in our example 98% of its tokens are input text. Put repeated text at the start of the prompt, so it is billed at the lower cached rate, and use a cheaper model.

Justus Amen · 9 min read

Meet EU requirements · GDPR & contracts

Zero-retention LLM APIs: what GDPR actually requires

Zero data retention (ZDR) describes a provider's retention commitment. The General Data Protection Regulation (GDPR) does not set one universal retention period for LLM APIs. It requires lawful processing, purpose limitation, data minimisation and deletion when personal data is no longer needed, with processor duties documented under Article 28.

Magnus Grünewald · 13 min read

Lower inference cost · Token & cache savings

How to cut LLM API costs without hurting output quality

To cut LLM API costs without hurting output quality, measure cost per request and a quality score on your own evaluation set first, then apply the levers in order of risk: cache repeated prompt starts, cap output and reasoning tokens, move suitable tasks to a cheaper open model, and route simple requests to a small model, re-running the same evaluation after each change and keeping only what holds the score. If your LLM API bill is growing faster than your usage, this guide ranks the levers by quality risk and prices each one at Lyceum Serverless Inference list rates.

Justus Amen · 11 min read

Keep your coding tools · Tool setup

How to use Lyceum models in goose

This guide shows how to run the goose agent on an open model from our serverless inference, with the endpoint split goose expects.

Maximilian Niroomand · 8 min read

Keep your coding tools · Tool setup

How to use Lyceum models in aider

To use a Lyceum model in aider, run aider on Python 3.12, set OPENAI_API_BASE to our OpenAI-compatible base URL and OPENAI_API_KEY to your Lyceum API key, then start it with --model openai/ followed by the Lyceum model id, for example openai/z-ai/glm-5.2. This guide covers the install, the endpoint, the config file, a cheaper editor model and a first edit on a git branch.

Maximilian Niroomand · 8 min read

Switch models safely · Open equivalents

Open-source vs proprietary LLMs in production: how to decide

Run a production workload on an open-weight model when that model clears your quality bar on your own evaluation set. You then choose the host, the licence and where prompts are processed. Keep a proprietary API where it gives the best tested combination of quality, latency, reliability, support, cost and contractual fit.

Caspar Lehmkühler · 12 min read

Lower inference cost · Unit economics

Inference cost per user: how to model AI feature unit economics

Inference cost per active user per month = interactions per user per month × tokens per interaction × blended price per token. If you are pricing an AI feature, this guide builds inference cost per user from a usage distribution, prices it at our serverless inference list rates and tests both total cohort margin and the cost of heavy users.

Justus Amen · 12 min read

Keep your coding tools · Tool setup

How to Use Lyceum Models in Kilo Code

This guide connects Kilo Code to an open-weight Lyceum model in VS Code or JetBrains, with the limits set correctly. Install the agent, add Lyceum as a custom OpenAI-compatible provider, set context and output limits in kilo.jsonc, and verify with one small task.

Maximilian Niroomand · 9 min read

Switch models safely · Migration plan

What Switching Inference Providers Actually Costs

Estimate a provider switch across evaluation, prompt changes, integration, parallel traffic and review work. The extra API bill depends on the new provider’s rates and actual mirrored tokens.

Maximilian Niroomand · 14 min read

Switch models safely · Evaluation

How to Read Open-Model Benchmarks Without Being Misled

Check who ran a benchmark, what it measures, its configuration and contamination risks. Then test candidates on your own workload, starting with a small pilot and expanding it before a production decision.

Caspar Lehmkühler · 12 min read

Keep GPU jobs running · Orchestration

Provisioning GPUs with Terraform: Provider & IaC Guide

Declare the GPU, driver and runtime separately. Prefer a native Terraform provider for VM lifecycle management. For an API-only service, keep an explicit operational boundary instead of treating a shell provisioner as a full provider.

Maximilian Niroomand · 12 min read

Model library · Publisher

Where to Run Qwen Models in Europe: The Full Line-Up Compared

Six Qwen models are listed as EU-hosted in Lyceum’s dashboard on 1 October 2026. Compare their prices, supported inputs and API identifiers, then test the model your workload needs.

Maximilian Niroomand · 10 min read

Lower inference cost · Token & cache savings

Cutting Inference Spend by Routing Requests by Difficulty

Some requests may work well on a smaller model. Measure that share, include verification and escalation costs, and test quality before routing production traffic.

Caspar Lehmkühler · 11 min read

Meet EU requirements · Transfer risk

Schrems II Training Data: Where Third-Country Risk Bites

A training pipeline can disclose personal data through storage, annotation, tracking, registries, compute and evaluation. Map each system and assess each disclosure against the EDPB transfer criteria.

Magnus Grünewald · 13 min read

Meet EU requirements · Provider vetting

EU-hosted LLM API providers: prices, regions and GDPR checks

An EU-hosted LLM API is one whose provider commits in writing, per endpoint and model, to process your prompts in Europe. Before you compare token prices, check that commitment, whether prompts are stored by default, and whether the model you need runs there. We read every provider page in this comparison on 20 October 2026.

Justus Amen · 13 min read

Meet EU requirements · Transfer risk

Is Your AWS EU Region Actually GDPR-Safe? The CLOUD Act Problem

An EU region helps establish processing location. Legal disclosure risk also depends on the entities and access involved. Assess both without assuming that a US parent automatically creates a GDPR transfer.

Magnus Grünewald · 15 min read

Run models without infra work · Agents & workloads

Multi-step agent workloads: latency and cost budgets

An AI agent makes many calls in a row, so one task can easily take half a minute and cost several cents. This guide shows how to give each task a time and cost limit and stay inside it: measure how many steps tasks really take, run independent tool calls in parallel, and stop at a step cap. In our worked example, that brought the slowest tasks from 34.9 to 28.1 seconds without changing the model. On Lyceum each step is billed per token, so the budget translates directly into your bill.

Maximilian Niroomand · 4 min read

Meet EU requirements · GDPR & contracts

Passing an Internal Security Review for a GPU Pilot

Prepare the data-flow description, processing-location statement, supplier evidence and exit plan for an internal GPU pilot review. The reviewer will assess the documents and the technical controls against your organisation’s requirements.

Justus Amen · 16 min read

Keep your coding tools · Team rollout

Piloting an Open-Model Coding Assistant Without Infrastructure

A credible pilot of an open-model coding assistant needs an endpoint, not a GPU: point Aider, Cline, Continue or Roo Code at a hosted OpenAI-compatible endpoint, agree criteria and a control first, and let per-token metering produce the evidence.

Maximilian Niroomand · 14 min read

Meet EU requirements · Data residency

AI Data Residency, Explained Properly

Data residency discussions combine storage, processing and operator jurisdiction. Check all 3, then assess the actual data flows and contract rather than treating an EU region as a compliance verdict.

Magnus Grünewald · 15 min read

Secure GPU capacity · Availability

B300 and GB200 Availability for Training Clusters in Europe

B300 and GB200 change cluster design through memory per device, interconnect, power and cooling, and the purchasing unit, not raw speed. This guide sizes a training job on both generations, shows where each part can actually be obtained in Europe, and gives the three questions that get a firm availability answer.

Justus Amen · 10 min read

Keep your coding tools · Agent compatibility

Testing Open Coding Models on Multi-File Agentic Edits

Test coding models on representative changes from your own repository. Use task-specific acceptance tests, regression checks and recorded costs for every attempt, including failures.

Maximilian Niroomand · 13 min read

Lower inference cost · Bill surprises

Setting Spend Limits on LLM Token Usage Across a Dev Team

If LLM spend across your team is only visible on the invoice, this guide puts attribution and an enforceable limit at the layer where they actually work.

Justus Amen · 12 min read

Keep your coding tools · Tool setup

How to Use Lyceum Models in Zed

This guide shows how to add an open-weight Lyceum model to Zed's agent panel with a single settings.json block. Learn how to correctly declare provider capabilities, specify context limits, and secure your API key.

Maximilian Niroomand · 8 min read

Avoid vendor lock-in · Provider alternatives

Together AI Alternatives for EU Data Residency

A roster of per-token providers with European processing, with the region question answered per provider and, where it matters, per model.

Magnus Grünewald · 11 min read

Model library · Publisher

Where to Run DeepSeek in Europe: V4 Flash vs Pro

If DeepSeek is blocked at your compliance review, this page answers per model where V4 Flash and V4 Pro run in Europe and what each costs.

Maximilian Niroomand · 10 min read

Lower inference cost · Serverless vs dedicated

Self-Hosting vs a Managed API: The Break-Even Volume

The usual comparison divides a GPU hourly rate by theoretical throughput and calls self-hosting cheaper. Priced with utilisation and the full serving stack included, the crossover moves, and a managed dedicated endpoint sits between the two extremes.

Justus Amen · 15 min read

Keep your coding tools · Tool setup

Roo Code OpenAI Compatible Provider: Lyceum Model Setup

This guide runs each Roo Code mode on an open-weight model from Lyceum Serverless Inference, chosen for that mode's job: one base URL and one key, a profile per mode, limits set by hand, and a tool-calling check before real work.

Maximilian Niroomand · 11 min read

Scale without limits · Reliability & support

Inference Provider Reliability: Verify Uptime Without an SLA

An SLA is a financial apology, not an engineering guarantee. Evaluate an inference provider's reliability by verifying their open-stack architecture, scrutinizing their public status page, and measuring latency metrics like TTFT and ITL yourself.

Justus Amen · 11 min read

Run models without infra work · Agents & workloads

RAG pipeline GPU sizing: cost per query, end to end

One question to a retrieval-augmented generation (RAG) assistant costs almost exactly its answer step. In our example, that is $0.00277 per query on our API, and two thirds of it pays for the retrieved text. On a GPU you rent, the same query costs between $0.0007 and $0.009, depending on how many queries share the GPU at once.

Caspar Lehmkühler · 11 min read

Secure GPU capacity · EU GPU providers

Lambda Labs Competitors in 2026: An Honest Rundown

Lambda Labs dominates US AI infrastructure, but rigid hourly pricing and a strictly US-based footprint push international teams to competitors. In 2026, European developers increasingly evaluate EU-sovereign alternatives for H100 capacity under strict GDPR compliance.

Magnus Grünewald · 10 min read

Meet EU requirements · AI Act

Article 50 AI Act: Marking Outputs with C2PA & SynthID

Article 50 of the EU AI Act imposes strict transparency obligations on generative media, splitting machine-readable marking from visible disclosure. Here is how C2PA, SynthID, and embedded metadata satisfy the rule, and why compliance is a pipeline decision you must own.

Magnus Grünewald · 12 min read

Meet EU requirements · AI Act

Provider or Deployer? AI Act Roles for Inference Engines

For teams building on a third-party inference engine, EU AI Act compliance starts with a counterintuitive fact: you are likely both a deployer of the upstream models and the provider of the AI system you ship.

Magnus Grünewald · 17 min read

Meet EU requirements · AI Act

Tamper-Evident AI Audit Trails: Hash Chains and Retention

A tamper-evident audit trail ensures any alteration to inference records is mathematically detectable. By building hash-chained logs over metadata and HMAC-SHA-256 digests in the application layer, you can prove system integrity without violating data retention limits.

Maximilian Niroomand · 15 min read

Meet EU requirements · AI Act

Is Your AI System High-Risk? A Decision Tree Through Annex III

Classifying your AI system under the EU AI Act is a rigid decision tree, not a judgement call. This guide maps out the Annex I and Annex III routes, breaking down the 4 conditions for derogation to give engineering teams a definitive exit state and compliance timeline.

Magnus Grünewald · 13 min read

Keep GPU jobs running · Distributed training

Training MoE Models: Preventing Expert Collapse & Balancing Load

Training a Mixture-of-Experts model introduces a critical bottleneck: if the router favors a small subset of experts, the starved parameters waste memory while overloaded ones halt the distributed run. Here is how to prevent routing collapse and balance MoE loads effectively.

Maximilian Niroomand · 15 min read

Meet EU requirements · AI Act

EU AI Act for Developers: A Practical Compliance Checklist (2026)

The EU AI Act assigns strict technical duties based on your role, but reading the legislation isn't practical. This routing hub indexes exactly which compliance obligations apply to your engineering team and links to the specific technical guides for implementation.

Magnus Grünewald · 9 min read

Keep GPU jobs running · Distributed training

Training Long-Context Models Without OOM: Ring Attention

When a training run fails on long sequences, the culprit is unsharded activation memory, not model parameters. Standard tensor and pipeline parallelism will not fix it. Context parallelism splits the sequence itself across GPUs, enabling massive contexts without OOMs.

Maximilian Niroomand · 11 min read

Meet EU requirements · AI Act

Does Fine-Tuning Make You a GPAI Provider?

Engineering teams worry fine-tuning an open-source model might classify them as a GPAI provider under the EU AI Act. By calculating compute against the Commission’s one-third threshold, you can prove your workload remains safely outside the scope.

Magnus Grünewald · 13 min read

Keep GPU jobs running · Fine-tuning & RL

Gradient Accumulation Loss Bug: Why Effective Batch Is Wrong

Gradient accumulation is assumed to be mathematically identical to full-batch training, but the standard implementation normalizes over the wrong denominator. Here is why the mean-of-means error skews weights, and how to verify if your fine-tuning setup is affected.

Maximilian Niroomand · 13 min read

Avoid vendor lock-in · Exit planning

Multi-Provider Inference Failover: Two Endpoints, One Codebase

If your product stops when one inference provider does, this guide puts a second endpoint behind the same code path. Learn how to configure multi-provider fallback, handle rate limits versus timeouts, and avoid breaking EU data residency during failover.

Maximilian Niroomand · 8 min read

Keep your coding tools · Tool setup

How to Use Lyceum Models in Cline

This guide shows how to run Cline's autonomous coding loop on an open-weight model from Lyceum Serverless Inference, with the limits set right.

Maximilian Niroomand · 6 min read

Keep your coding tools · Tool setup

How to Use Lyceum Models in GitHub Copilot

GitHub Copilot now supports custom endpoints, allowing development teams to use open-weight models via bring-your-own-key. This guide explains how to configure VS Code to connect to Lyceum Serverless Inference, bypass model name restrictions, and avoid empty-response errors.

Maximilian Niroomand · 9 min read

Meet EU requirements · AI Act

Annex IV Technical Documentation: The ML Team Checklist

Annex IV of the EU AI Act turns technical documentation into a strict legal requirement for high-risk AI systems. This guide translates the 9 mandatory legal points into a concrete checklist for ML engineering teams.

Magnus Grünewald · 14 min read

Meet EU requirements · AI Act

Do You Need a DPIA for LLM Inference? A Deployer's Guide

Before shipping an LLM feature, you need to know if sending prompts to an API triggers a DPIA. This guide clarifies that the DPIA is a GDPR instrument, not an AI Act one, and maps exactly how to extract the 4 mandatory compliance inputs from your inference provider.

Magnus Grünewald · 16 min read

Keep GPU jobs running · Memory & OOM

Sequence Packing: Reclaiming GPU Hours in Long-Context SFT

Padding waste silently inflates long-context SFT costs by processing empty tokens. This guide explains how to calculate token occupancy, migrate from length-grouped batching to true sequence packing, and prevent the three silent correctness bugs that destroy model quality.

Maximilian Niroomand · 13 min read

Model library · Launches

DeepSeek-V4.1-Flash: specs, benchmarks, and how to run it

DeepSeek-V4.1-Flash is live on Lyceum Serverless Inference: this page covers what changed, what it costs per token ($0.50 input, $0.13 cached, $1.50 output per 1M) and the exact request to send. Architecture and benchmark figures are DeepSeek's own, vendor-reported.

Maximilian Niroomand · 7 min read

Keep your coding tools · Agent compatibility

Empty Response on a Reasoning Model: Why and How to Fix It

A blank reply can result from an output limit, a client integration problem or a response that needs further handling. Inspect the complete response before changing settings. Reasoning text alone is not a final answer.

Maximilian Niroomand · 11 min read

Avoid vendor lock-in · Provider alternatives

How AI Consultancies Choose LLM APIs for Client Projects

For AI consultancies, selecting an LLM API is about managing deal risk and reselling margins. This guide breaks down how to protect client data, avoid vendor lock-in with OpenAI SDK compatibility, and deploy EU-sovereign models to pass strict enterprise InfoSec audits.

Justus Amen · 10 min read

Switch models safely · Migration plan

Switching the OpenAI SDK to an Open-Model Endpoint

Changing the base URL in the OpenAI SDK takes a minute, but a true migration requires checking five critical behavioural differences underneath the compatible interface. This guide covers how to repoint the SDK and verify structured output, tool calls, and ignored parameters.

Maximilian Niroomand · 10 min read

Scale without limits · Latency & throughput

Time to first token (TTFT): what drives LLM API latency

Time to first token (TTFT) is how long an LLM API takes to start its reply. It adds network travel, time waiting in a queue, and the time the model spends reading your prompt. Our serverless API reuses repeated prompt starts automatically, which cut that wait by about 45% in our test.

Maximilian Niroomand · 10 min read

Switch models safely · Evaluation

Is Your Inference Provider Quantizing the Model: How to Tell

If a model behaves differently across providers, compare task quality under controlled settings. These tests cannot prove quantization; request documented serving details to understand the configuration.

Justus Amen · 10 min read

Keep GPU jobs running · Distributed training

Multi-Node GRPO Orchestration: Ray, Slurm or Kubernetes

When scaling GRPO to multi-node clusters, Ray is not an alternative to Slurm or Kubernetes; it is the runtime that sits inside them. Discover why RL's co-dependent architecture makes gang scheduling non-negotiable and how to orchestrate your training jobs on Lyceum.

Maximilian Niroomand · 13 min read

Keep GPU jobs running · Reliability

Checkpoint Resume Loss Spikes: Fixing Optimizer & LR State

When a long training run is interrupted, resuming from a checkpoint often triggers a massive loss spike. The most common culprits are missing optimizer moment buffers, reset learning rate schedulers, or mismapped FSDP shards - here is the triage order and recovery checklist.

Maximilian Niroomand · 13 min read

Meet EU requirements · Provider vetting

Sovereign-Washing: How to Vet a "Sovereign" EU Cloud Claim

Cloud providers routinely claim EU sovereignty without removing non-EU legal or operational dependencies. Here is an eight-question framework to cut through sovereignty washing, followed by an honest self-assessment of where we pass and where we fall short.

Magnus Grünewald · 13 min read

Keep your coding tools · Tool setup

How to Use Lyceum Models in Cursor

Connect Cursor to Lyceum through its OpenAI-compatible endpoint, then test the model in Ask/chat. Agent support depends on the provider, model and client version. This guide separates the documented setup from features that still need validation.

Maximilian Niroomand · 8 min read

Keep GPU jobs running · Distributed training

How Many GPUs for Trainer vs Rollout in RL Post-Training

In a disaggregated reinforcement learning pipeline, colocation is obsolete. Here is how to instrument your RL post-training loop, measure phase-level timings, and properly split your GPU fleet between the compute-bound trainer and the memory-bound rollout engine.

Maximilian Niroomand · 13 min read

Avoid vendor lock-in · Exit planning

Self-Hosting vs Managed EU Inference: The Independence Trade-Off

Compare processing location, provider access, portability and operational control before choosing managed inference or self-hosting. Match the deployment and contract to your actual requirements.

Magnus Grünewald · 7 min read

Keep GPU jobs running · Fine-tuning & RL

What It Costs to Train a Text-to-Video Model: Real GPU Budgets

Calculate the real cost of a text-to-video training run using active parameters, latent tokens, and dense MFU. Build a realistic campaign budget in GPU-hours to multiply by your own quoted hardware rates.

Justus Amen · 11 min read

Meet EU requirements · AI Act

AI Act Article 26: Deployer Logging & 6-Month Retention Explained

For high-risk AI deployers, Article 26(6) requires keeping system logs for at least six months. When using a zero-retention API, the provider stores nothing, meaning this logging capability must be built entirely within your own application.

Magnus Grünewald · 13 min read

Keep GPU jobs running · Fine-tuning & RL

GRPO vs DPO vs RLHF: Compute Cost and GPU Footprint Compared

Direct Preference Optimization (DPO), RLHF, and GRPO scale their compute footprints differently based on how many models they keep resident. Here is the definitive comparison of resident model headcounts, generation wall-clock time, and memory bandwidth constraints.

Maximilian Niroomand · 14 min read

Switch models safely · Open equivalents

DeepSeek V4 Pro vs Claude Opus 4.6: Agent Costs Compared

If you are weighing DeepSeek-V4-Pro against Claude Opus 4.6, this page gives the verified per-token comparison on your own token ratio. We show how agent workflows accumulate context and why you must evaluate cost at the finished task level before switching.

Maximilian Niroomand · 12 min read

Model library · Changelog

EU Hosted LLM API: Lyceum Model Roster, Prices and Changes

Which model strings answer on this OpenAI-compatible endpoint, which offering records publish a hosting location and complete price data, and which older strings are absent from the live roster. Facts checked on 10 September 2026, with a roster request you can rerun yourself.

Caspar Lehmkühler · 10 min read

Lower inference cost · Serverless vs dedicated

Per-Token vs Per-GPU-Hour: Which Inference Pricing Fits

If you are choosing between paying per token and paying per GPU-hour, this guide reframes it as a question about who carries the utilisation risk and matches each option to a traffic shape.

Justus Amen · 12 min read

Lower inference cost · Price comparisons

Qwen3-235B Cost per Token Across Providers

If the same Qwen3-235B is priced very differently across providers, this page names the five dimensions behind the gap and shows how to compare like for like.

Justus Amen · 12 min read

Lower inference cost · Serverless vs dedicated

Self-Host vs API: The Token Break-Even for Open Models

If you are applying one self-hosting rule of thumb to every model you run, this guide shows how the crossover moves with model size and where it reverses entirely.

Justus Amen · 10 min read

Keep your coding tools · Agent compatibility

Self-Hosting Code Completion for VS Code and JetBrains

Transitioning from closed ecosystems to a VS Code open source code completion model guarantees data sovereignty for European engineering teams. By pairing a local IDE extension with an EU-hosted serverless inference endpoint, developers achieve low-latency coding assistance.

Maximilian Niroomand · 9 min read

Keep your coding tools · Agent compatibility

Verify Coding Agent Function Calling Before Switching

When migrating an AI coding agent to a sovereign inference engine, ensuring OpenAI-compatible function calling is critical. Learn how to test tool calling support on open-weight models to avoid broken workflows.

Maximilian Niroomand · 13 min read

Keep your coding tools · Team rollout

What a Works Council and DPO Ask Before an AI Coding Tool Ships

Before an AI coding tool ships, IT compliance, Data Protection Officers, and Works Councils must approve the deployment. Here is exactly what they ask about pattern learning, employee monitoring, and data residency, and how to build an approval package that satisfies them.

Magnus Grünewald · 12 min read

Run models without infra work · Agents & workloads

AI dubbing costs: GPU pipelines vs commercial APIs

Dubbing as one API costs $2.20 per minute on ElevenLabs Dubbing v2, read on 30 October 2026. Bought stage by stage, transcription, translation and the synthetic voice cost about $0.10 per minute in our example. The extra $2.10 pays for automation: detecting speakers, keeping their voices and fitting the new speech into the video. On Lyceum you can run translation and transcription yourself in European data centres: translation on our API, with prompts and outputs not stored, and transcription as a GPU batch job billed only while it runs.

Justus Amen · 11 min read

Lower inference cost · Price comparisons

Cheapest Way to Run DeepSeek V4 via API

Finding the cheapest DeepSeek V4 API starts by recognizing that V4 is actually two models: Pro and Flash. Before comparing provider rates, you must choose your variant and understand how input and output splits drive your true per-token cost.

Maximilian Niroomand · 9 min read

Keep GPU jobs running · Fine-tuning & RL

GRPO VRAM & GPU Sizing: Policy, Reference and Reward

If a GRPO run is running out of memory, this guide accounts for every model copy the algorithm keeps resident. Sizing GPUs correctly requires accounting for the policy's optimiser state, inference copies, and the rollout cache before you start.

Justus Amen · 12 min read

Model library · Coding

Kimi-K2.7-Code: code-specialised Kimi, EU-hosted

Kimi K2.7 Code is Moonshot's coding model, and one request holds about 32,000 lines of Python when the reply is short. We serve it as moonshotai/kimi-k2.7-code for $1.25 per million tokens sent and $4.50 per million written (23 October 2026). You keep your OpenAI SDK, pay only per token, and your code is processed in Paris or Finland. We don't keep it.

Maximilian Niroomand · 11 min read

Switch models safely · Task fit

Kimi-K3 vs DeepSeek-V4-Pro for Reasoning Work

Comparing Kimi-K3 and DeepSeek-V4-Pro solely on per-token price is misleading for reasoning work. Because models emit massive volumes of intermediate thought tokens, the true cost metric is the total billed volume per finished, correct answer.

Maximilian Niroomand · 12 min read

Model library · Text

MiniMax-M3 EU-hosted: price, limits and the move from M2.5

MiniMax-M3 runs EU-hosted on our serverless inference, a pay-per-token API for open models that works with the OpenAI SDK and never stores your prompts. It costs $0.40 per million input tokens and $2.00 per million output tokens (23 October 2026). MiniMax-M2.5 is retired, so you move to M3 by changing the model string.

Maximilian Niroomand · 10 min read

Switch models safely · Open equivalents

MiniMax M3 vs Claude Sonnet 5: The Mid-Tier Cost Gap

If you are weighing MiniMax-M3 against Claude Sonnet 5, this page gives the verified per-token comparison on your own token ratio and what you would need to test before switching.

Maximilian Niroomand · 10 min read

Keep GPU jobs running · Distributed training

Agentic RL GPU Cost: Multi-Turn Rollouts and KV Reuse

Agentic reinforcement learning repeats identical prefill calculations across turns and rollouts, compounding GPU hours. Moving a serving engine like vLLM into the training loop stops this waste by reusing the KV cache across the entire group.

Justus Amen · 11 min read

Model library · Text

GLM-5.2 Instant: Latency-Optimised GLM, EU-Hosted

GLM-5.2 Instant is GLM-5.2 with its reasoning phase switched off, so it answers straight away at the same per-token price. Use it in coding tools such as GitHub Copilot and Cursor, which limit answer length and hide the model's thinking. You set it as the model on our OpenAI-compatible API, and your prompts are processed in Paris or Finland. We don't keep them.

Maximilian Niroomand · 8 min read

Switch models safely · Open equivalents

GLM-5.2 vs Claude Opus 5: The Real Cost Gap

For enterprise teams comparing GLM-5.2 against Claude Opus 5, the true cost difference depends entirely on your token ratio. This guide breaks down the per-token math, where each model processes your data, and what to test before migrating your workload.

Maximilian Niroomand · 11 min read

Model library · Text

Qwen3.5-9B: Compact Qwen With the Narrowest Price Spread

Qwen3.5-9B is a compact open-weight model with a 256K context window and the narrowest input-to-output price spread in the catalogue. Costing $0.15 per million input tokens and $0.20 per million output tokens, it significantly reduces the total bill for output-heavy workloads.

Maximilian Niroomand · 9 min read

Run models without infra work · Agents & workloads

Running Wan 2.2 on cloud GPU: VRAM needs & cost per clip

Wan 2.2's large video model, with 14 billion parameters, needs a GPU with 80 GB of memory to render 720p clips. On one NVIDIA H100, a five-second 720p clip took the Wan team 1,041.5 seconds. At our on-demand H100 price, that comes to about $1.36 per clip.

Maximilian Niroomand · 11 min read

Run models without infra work · Custom deployments

Static IP and DNS for GPU inference endpoints

Enterprises often ask an AI provider for a static IP, a network address that never changes, so they can write a firewall rule for it. Most managed inference endpoints don't have one, because the provider moves servers. A firewall rule on the provider's hostname, plus an API key, does the same job and keeps working when the address changes. Separately, if your company traffic runs through a proxy, turn on streaming for long answers, or the proxy may cut them off.

Maximilian Niroomand · 12 min read

Lower GPU cost · Billing models

Predictable GPU Cloud Pricing: How to Avoid Surprise Bills

A GPU cloud invoice is rarely just the hourly compute rate. By bounding egress fees, zombie storage, and idle replicas, engineering teams can make their next infrastructure bill entirely predictable before they provision a single node.

Justus Amen · 12 min read

Meet EU requirements · Provider vetting

Sovereignty Beyond Hosting: Why an EU Region Isn't Enough

AI sovereignty requires more than selecting an EU server location in a hyperscaler console. True independence means controlling your processing location, model weights, commercial terms, and technical stack to ensure complete autonomy over your infrastructure.

Magnus Grünewald · 9 min read

Keep your coding tools · Endpoint swap

Using Lyceum Models in opencode: Custom Provider Setup

opencode supports custom OpenAI-compatible endpoints via its provider configuration. Pointing it at a European serverless inference endpoint lets developers use frontier open-weight models directly in the terminal, cutting token costs while keeping codebase data inside the EU.

Caspar Lehmkühler · 11 min read

Meet EU requirements · Provider vetting

Recommending an EU Inference Provider: A Deal-Risk Checklist

If you are about to recommend an inference provider to a client, this checklist covers the four risks that land on you rather than on them - and applies itself to us.

Justus Amen · 13 min read

Lower GPU cost · Billing models

Base-Fee vs Usage-Only GPU Pricing Compared

The decision between base-fee and usage-only GPU pricing is dictated by your workload's duty cycle, not the headline hourly rate. Before comparing providers, calculate your effective cost per hour and identify the hidden fees that keep the meter running at low utilization.

Justus Amen · 12 min read

Run models without infra work · Agents & workloads

GPU cost for batch vision inference: DINOv3, SAM & CLIP

Running DINOv3, SAM or CLIP over millions of images gets expensive when the GPU sits idle, waiting for the CPU to decode and resize the next images. You pay for every GPU hour, busy or not. So before you rent a bigger GPU, check whether yours is waiting, and speed up image loading first. On Lyceum you rent GPU machines by the second, and spot machines cost about half as much, which suits a batch job that can restart.

Justus Amen · 12 min read

Pick the right GPU · GPU head-to-head

AMD MI300X vs H100 for Training: ROCm Maturity & Real Throughput

To evaluate the AMD MI300X for training, teams must look past paper compute and measure the ROCm software tax. The 192 GB memory offers massive batching advantages, but realizing throughput demands kernel tuning, and single-node parity does not guarantee multi-node scaling.

Maximilian Niroomand · 13 min read

Model library · Launches

GLM-5.3 Flash: specs, benchmarks, and how to run it on Lyceum

GLM-5.3 Flash is Z.ai's low-cost GLM model for coding and agent work, and it reads text and images. We serve it EU-hosted as z-ai/glm-5.3-flash for $0.20 per million input tokens and $0.50 per million output tokens. You keep your OpenAI SDK, pay only per token, and your prompts are processed in Paris or Finland. We don't keep them.

Maximilian Niroomand · 9 min read

Model library · Launches

GLM-5.3: specs, benchmarks, and how to run it on Lyceum

GLM-5.3 is Z.ai's larger GLM model for long coding and agent tasks. We serve it EU-hosted as z-ai/glm-5.3 for $1.40 per million input tokens, $0.26 for repeated input and $4.40 for output. You call it with your OpenAI SDK, pay per token with no base fee, and your prompts are processed in Paris or Finland. We don't keep them.

Maximilian Niroomand · 9 min read

Switch models safely · Task fit

Open Models with Reliable Function Calling & JSON Output

Discover why reliable JSON output is more than just picking a model off a leaderboard. Learn how to combine open models, inference-engine constraints like guided decoding, and tiered retry logic to build cost-effective function calling pipelines.

Caspar Lehmkühler · 12 min read

Model library · Launches

Qwen3.8 2.4T A95B: specs, benchmarks, and how to run it on Lyceum

This page is for engineers sizing GPUs or an API budget for Qwen3.8 2.4T A95B. It is an open-weight model from Qwen with 2.4 trillion parameters (its learned numbers), of which 95 billion work on each token. On Lyceum you call it through an OpenAI-compatible API, billed per token: $2.50 per million input tokens, $6.00 per million output tokens. Running the weights yourself needs a cluster of data-center GPUs; the table in the hardware section gives the counts.

Maximilian Niroomand · 10 min read

Model library · Launches

Qwen3.8 27B: specs, benchmarks, and how to run it on Lyceum

Qwen3.8 27B is Qwen's compact open model, and it reads text and images. We serve it EU-hosted as qwen/qwen3.8-27b for $0.40 per million input tokens and $2.40 per million output tokens. You call it with your OpenAI SDK, pay per token, and can also fine-tune it with us in early access.

Maximilian Niroomand · 9 min read

Model library · Launches

Qwen3.8 Flash Next: specs, benchmarks and Lyceum API

Qwen3.8 Flash Next is a low-cost Qwen model that reads text and images. We serve it EU-hosted as qwen/qwen3.8-flash-next for $0.20 per million input tokens and $0.50 per million output tokens (23 October 2026). You keep your OpenAI SDK, pay only for the tokens you use, and your prompts are processed in Paris or Finland. We don't keep them.

Maximilian Niroomand · 10 min read

Scale without limits · Latency & throughput

Text Embedding API: Throughput, Batching and Cost

Embedding inference is prefill-only, fundamentally changing how workloads scale. Size your corpus backfill and live query path separately, eliminate padding waste, and decide between serverless and dedicated endpoints based on duty cycle rather than instinct.

Maximilian Niroomand · 12 min read

Pick the right GPU · GPU head-to-head

NVFP4 Training on Blackwell: The Recipe & Throughput

NVIDIA's NVFP4 format enables 4-bit pretraining on Blackwell, but real throughput depends on the precision of your master weights and scaling overhead. We break down the NVFP4 pretraining recipe, real Blackwell TFLOPS, and how to validate convergence before committing to a term.

Maximilian Niroomand · 12 min read

Switch models safely · Open equivalents

OpenAI & Anthropic Replacements: Open-Model Map

A task-by-task migration map for teams replacing OpenAI or Anthropic APIs with open-weight models. We cover the exact models, per-token prices, and hosting regions to match your specific workloads.

Maximilian Niroomand · 12 min read

Scale without limits · Latency & throughput

Prefill/Decode Disaggregation: Faster Long-Context LLM Serving

Prefill-decode disaggregation splits compute-heavy prompt processing from memory-bound token generation onto separate GPU pools. It eliminates the latency spikes caused when long contexts stall active decodes, optimizing SLO adherence without sacrificing hardware utilization.

Maximilian Niroomand · 12 min read

Avoid vendor lock-in · Exit planning

AI Pilot Exit Criteria: Making Reversibility a Requirement

A staggering of enterprise generative AI pilots fail to deliver measurable business impact. Defining explicit exit criteria and choosing a reversible infrastructure stack ensures you can stop a proof of concept cleanly without stranded costs or vendor lock-in

Justus Amen · 10 min read

Run models without infra work · GPU efficiency

Fixing vLLM CUDA out of memory: KV cache tuning guide

If you run an open model on your own GPUs with vLLM, the server can stop with "CUDA out of memory". It hits teams that host large models such as Llama 3.1 70B themselves. It is rarely a leak: at startup vLLM reserves GPU memory for the model and for the requests it will handle, and the two don't fit. If it fails at startup, give the model more room with a second GPU or an 8-bit version. If it fails under load, too many long requests share the memory, so shorten them or run fewer at once.

Maximilian Niroomand · 13 min read

Scale without limits · Latency & throughput

Speculative Decoding: Acceptance Rate vs. Throughput

Speculative decoding trades spare memory bandwidth for faster token generation, but at high concurrency, it competes with real requests and slows down throughput. Here is how to calculate your acceptance rate and find the exact concurrency where your GPU stops being memory-bound.

Maximilian Niroomand · 13 min read

Lower GPU cost · Expired credits

When AWS or GCP Credits Expire: AI Migration Playbook

When startup cloud credits expire, AI product companies face a sudden surge in infrastructure costs. Moving to open-weight models on serverless inference cuts per-token spend sharply and lets you pick models hosted in European data centres.

Maximilian Niroomand · 10 min read

Scale without limits · Cold starts & autoscaling

Autoscaling GPU Inference: KServe vs Ray Serve vs llm-d

KServe, Ray Serve, and llm-d offer different approaches to scaling GPU inference on Kubernetes. While KServe standardizes general model serving and Ray Serve enables Python-native pipelines, llm-d adds LLM-specific optimizations like disaggregated prefill and decode.

Maximilian Niroomand · 12 min read

Avoid vendor lock-in · Provider alternatives

EU Alternatives to AWS Bedrock and Azure OpenAI

AWS Bedrock and Azure OpenAI offer enterprise familiarity, but hidden egress fees and US CLOUD Act exposure drive up costs and compliance risks. EU-sovereign alternatives deliver strictly GDPR-compliant, OpenAI-compatible infrastructure without the hyperscaler tax.

Magnus Grünewald · 10 min read

Switch models safely · Task fit

Best Open-Model APIs for Agentic Coding (2026)

Agentic coding fundamentally changes model economics, shifting the focus from single-shot completions to multi-step tool calls where output prices compound. This guide breaks down the 18-fold output price spread across open models for autonomous agents.

Maximilian Niroomand · 12 min read

Switch models safely · Task fit

Best Open Vision-Language Model APIs (2026)

For enterprise AI teams, evaluating open vision-language models comes down to balancing reasoning depth, inference cost, and data residency. Here is a direct comparison of the top EU-hosted multimodal APIs, Qwen2.5-VL and MiniCPM-V 4.5, and how to test them on your payloads.

Caspar Lehmkühler · 10 min read

Switch models safely · Task fit

GLM-5.2 vs Kimi-K2.6 vs Qwen3: Coding APIs Compared

Comparing GLM-5.2, Kimi-K2.6, and Qwen3-Coder-30B-A3B reveals a clear divide: two are general-purpose flagships for complex reasoning, and one is a highly distilled code specialist. We break down the architectures, use cases, and the twenty-fold price gap between them.

Maximilian Niroomand · 9 min read

Switch models safely · Task fit

Best Open Model API for OCR and Document Extraction

Vision-language models have made traditional OCR obsolete by extracting structured JSON directly from document images. For European teams, running these models on an EU-hosted, zero-retention API solves the GDPR compliance challenge of processing invoices and contracts.

Caspar Lehmkühler · 11 min read

Switch models safely · Task fit

Best Open Model for RAG Generation: Which Size Wins

When building a RAG pipeline, the generation model acts as a reading comprehension engine rather than a factual knowledge base. Discover why choosing an efficient 30B model over a massive 235B architecture slashes your compute bill while delivering the exact same answers.

Caspar Lehmkühler · 11 min read

Lower GPU cost · Rent vs own

What a GPU Cluster Quote Should Contain Before You Sign

Evaluating a GPU cluster quote requires looking beyond the hourly hardware rate. This guide breaks down the essential technical criteria, from network fabric and node-level SLAs to hidden TCO exclusions, that engineering teams must validate before signing a contract.

Justus Amen · 13 min read

Keep GPU jobs running · Distributed training

Weight Sync Between Trainer and vLLM: The Hidden Cost in RL Loops

Moving updated policy weights from your trainer to vLLM for rollouts incurs a recurring time penalty that bleeds capital. This guide models the true cost of weight synchronization over disk, NCCL, and delta transfers, explaining how your cluster fabric sets the limit.

Maximilian Niroomand · 12 min read

Avoid vendor lock-in · Exit planning

AI Vendor Risk Assessment: A Procurement Checklist

Standard third-party risk questionnaires miss AI-specific vulnerabilities like model data retention and training rights. Here is the exact checklist procurement teams need to vet AI infrastructure vendors, complete with our own honest answers.

Justus Amen · 13 min read

Switch models safely · Task fit

30B vs 70B vs 235B: How to Pick Open Model Size Per Task

Parameter count is no longer a reliable proxy for inference cost. With Mixture-of-Experts architectures breaking the linear pricing curve, you can stop guessing and use a simple per-token price ladder to size open models precisely against your workload.

Caspar Lehmkühler · 11 min read

Switch models safely · Task fit

Best Multilingual Embedding APIs for RAG (2026)

Choosing the right multilingual embedding API requires testing on your own corpus rather than trusting aggregate leaderboard scores. Here is how to evaluate retrieval quality across languages, avoid silent vector mismatches, and leverage Lyceum's EU-hosted Qwen3-Embedding-8B.

Caspar Lehmkühler · 11 min read

Meet EU requirements · GDPR & contracts

The DPA Question: Sub-Processors in AI Inference

An inference provider's data processing agreement must say who else handles your client's data and warn you before that changes. Ask for the named list and the notice clause before you go to production. Our DPA is part of our terms, and we send the list on request.

Justus Amen · 11 min read

Meet EU requirements · Transfer risk

Can You Use US-Based AI APIs and Stay GDPR Compliant?

Sending API prompts to US-based AI models exposes European enterprises to severe GDPR compliance risks. True data sovereignty requires avoiding cross-border transfers entirely by processing the 3 tiers of personal data exclusively on EU-hosted infrastructure.

Magnus Grünewald · 11 min read

Avoid vendor lock-in · Exit planning

Porting Fine-Tunes and LoRA Adapters Between Providers

The true value of your fine-tune is the knowledge embedded in its weights. By extracting your LoRA adapters as portable artefacts and avoiding proprietary serving layers, you can freely migrate your custom models across any infrastructure without vendor lock-in.

Maximilian Niroomand · 11 min read

Model library · Publisher

Running GLM 5.1, 5.2 and 5.2 Instant in Europe: self-hosting and serverless options

You can run GLM in Europe in two ways. You host the open weights yourself, which for GLM-5.2 takes eight NVIDIA H200s, data-centre GPUs with 141 GB each. Or you call GLM-5.2, GLM-5.2 Instant or GLM-5.3 on our OpenAI-compatible API in Paris and Finland, paying per token with no servers to run.

Maximilian Niroomand · 10 min read

Meet EU requirements · Data residency

Which open-weight models are actually hosted in Europe, and where

Every model on our models page runs on EU-hosted infrastructure, as each model's own page showed on 23 October 2026. Our serverless models run in Paris and Finland, and we don't keep your prompts or use them for training. So you can document residency per model ID from one page each.

Magnus Grünewald · 8 min read

Secure GPU capacity · EU GPU providers

European GPU Cloud Providers Compared

As hyperscaler credits expire and the EU AI Act takes effect, AI scaleups are moving workloads to specialized European infrastructure. We compare the leading European GPU cloud providers on sovereignty, egress costs, and hardware ownership.

Magnus Grünewald · 13 min read

Meet EU requirements · GDPR & contracts

Zero Data Retention in LLM Inference: How to Verify It

Zero data retention means a provider keeps no copy of your prompts or its answers once a request is done. You check it in the contract and in the provider's account of each step a request passes. We never log your prompts or write them to a database.

Magnus Grünewald · 10 min read

Model library · Text

DeepSeek-V4-Flash: specs, size, benchmarks and how to run it

DeepSeek-V4-Flash is an open-weight model with 284 billion parameters, of which only 13 billion run for each token, and a context of up to 1M tokens. It beats the V4-Pro preview on DeepSeek's coding and agent benchmarks. On our EU-hosted API it costs $0.25 input, $0.06 cached input and $0.30 output per million tokens.

Maximilian Niroomand · 9 min read

Model library · Reasoning

DeepSeek V4 Pro API: EU hosting, pricing and context limits

You can call DeepSeek V4 Pro through the Lyceum OpenAI-compatible API, EU-hosted, as deepseek/deepseek-v4-pro-0813. It costs $2.00 per million input tokens and $4.00 per million output tokens, and one request can read up to 1M tokens. We don't keep your prompts or outputs, and we never use them for training.

Maximilian Niroomand · 8 min read

Secure GPU capacity · Availability

GPU Lead Times: Realistic Expectations and Provider Questions

The 2026 compute landscape is defined by scarcity, with memory constraints pushing cloud GPU lead times to 52 weeks. Here is how to navigate availability guarantees, avoid hyperscaler idle-compute waste, and ask the right questions to secure sovereign EU infrastructure.

Magnus Grünewald · 10 min read

Switch models safely · Evaluation

How to Test an Open-Weight Model for Free Before You Commit

Evaluating open-weight models on free API tiers allows teams to benchmark latency, cost, and quality without hardware capex. By pairing free trial credits with an automated evaluation harness, engineers can validate an LLM's performance on domain-specific tasks before committing.

Caspar Lehmkühler · 13 min read

Avoid vendor lock-in · Exit planning

Model Deprecation Risk: Version Pinning & Notice Periods

Model deprecation risk is the chance that the model your product calls is retired or changed, and your app breaks or behaves differently. You lower it in three ways: call an exact model ID, keep a test set of your own prompts, and know your provider's notice period.

Caspar Lehmkühler · 7 min read

Lower inference cost · Seat licences

Per-Seat Licences vs Per-Token Inference: Where the Line Sits

For enterprise AI, the math is shifting from per-seat licences that start at $30 per user per month to consumption-based inference. Transitioning to per-token open models scales AI usage without artificially inflating headcount costs, provided you control the output-token tax.

Magnus Grünewald · 11 min read

Secure GPU capacity · Availability

On-Demand GPUs Sold Out? Where to Find Capacity Fast

When an on-demand GPU request fails, engineers need a same-day triage path to keep workloads moving. This guide breaks down how to bypass waitlists, validate quota limits, adapt models to available hardware, and secure compute capacity fast.

Magnus Grünewald · 11 min read

Secure GPU capacity · Provider trust

Reading GPU Cloud Provider Reviews: Uptime Signals

Aggregated vendor reviews rarely highlight the infrastructure metrics that matter most for production workloads. This guide unpacks how to evaluate GPU cloud SLAs, status pages, and capacity guarantees to ensure true reliability for your AI infrastructure.

Magnus Grünewald · 12 min read

Secure GPU capacity · Reservations

Reserved vs On-Demand GPUs: True Capacity Guarantees

A GPU reservation is often treated as a pure cost-saving measure, but its true value is mitigating availability risk. We examine what SLA capacity guarantees actually commit providers to, the failure modes hidden in the fine print, and when on-demand remains the safer choice.

Magnus Grünewald · 12 min read

Secure GPU capacity · Provider head-to-head

RunPod vs Vast.ai: Which GPU Marketplace Fits Which Workload

Choosing between RunPod and Vast.ai comes down to the trade-off between managed infrastructure and peer-to-peer pricing. While Vast.ai offers rock-bottom rates via an auction marketplace, RunPod provides predictable tiers and serverless execution for production pipelines.

Magnus Grünewald · 13 min read

Secure GPU capacity · Provider trust

Vast.ai Reliability Depends on the Host: Renting Checklist

Vast.ai offers some of the lowest listed GPU rates on the market, but its decentralized structure means uptime varies wildly by host. Before moving workloads from a managed cloud, engineering teams must evaluate verification scores, checkpointing overhead, and data residency.

Magnus Grünewald · 12 min read

Secure GPU capacity · Availability

What Limits GPU Availability: HBM, CoWoS and Power

The true bottleneck for AI capacity has moved from the silicon foundry to advanced packaging and the local power grid. Here is a breakdown of the physical supply chain gating GPU availability, and how to identify what is actually deployable.

Maximilian Niroomand · 12 min read

Lower inference cost · Bill surprises

How Lyceum's Serverless Inference Billing Works

Lyceum's billing model is built to eliminate idle waste and hidden networking fees. By combining pay-per-token Serverless Inference with per-second workload execution and zero egress charges, it ensures you only pay for the exact compute and tokens your models use.

Caspar Lehmkühler · 9 min read

Keep your coding tools · Tool setup

How to Use Lyceum API within Claude Code

This guide shows how to run Claude Code on an open-weight model through Serverless Inference, with three commands and no proxy, and how to check which model is answering.

Caspar Lehmkühler · 8 min read

Switch models safely · Open equivalents

Kimi K3 vs Claude Fable 5: The Top-Tier Comparison

Kimi K3 matches Claude Fable 5's top-tier reasoning with 2.8 trillion parameters and a 1-million-token context window, all at a lower list price. European AI teams can run Kimi K3 on Lyceum's eu-north1 infrastructure for full data sovereignty

Magnus Grünewald · 6 min read

Model library · Publisher

Where to run Kimi models in Europe: Kimi K3 and K2.7 Code

We serve two Kimi models from Europe, Kimi K3 and Kimi K2.7 Code, on GPUs in Paris and Finland. You call both through one API that works with the OpenAI SDK, pay per token, and prompts are never stored or used for training. Kimi K2.6 is retired, Kimi K3 is its replacement.

Maximilian Niroomand · 7 min read

Lower inference cost · Unit economics

Finding the Cheapest Open Model That Clears Your Quality Bar

Most teams default to the largest models available, driving up inference bills unnecessarily. By defining a strict quality bar and testing from the cheapest open model upward, you can drastically reduce compute costs without sacrificing output quality.

Maximilian Niroomand · 8 min read

Lower inference cost · Bill surprises

How to Estimate Serverless Inference Costs Before You Commit

Provider quotes for serverless inference are difficult to compare. By understanding the core identity that converts throughput into cost per token, you can evaluate quotes against your own workload's batching, quantization, and utilisation metrics.

Caspar Lehmkühler · 13 min read

Avoid vendor lock-in · Provider alternatives

Hugging Face Inference Endpoints Cost vs Serverless GPU

Hugging Face Inference Endpoints bill by the instance hour, meaning you pay for uptime instead of actual usage. For low-traffic APIs, an always-on endpoint is an expensive overspend. We analyze the duty-cycle crossover where serverless GPUs become the cheaper choice.

Maximilian Niroomand · 10 min read

Lower inference cost · Price comparisons

Image Generation API Pricing: Cost Per Image Compared

Per-image pricing hides the real cost drivers of generative AI: diffusion steps and resolution. This guide breaks down how to calculate true cost per image, compares leading API providers, and proves exactly when a dedicated GPU mathematically beats pay-as-you-go billing.

Maximilian Niroomand · 10 min read

Avoid vendor lock-in · Provider alternatives

Modal vs RunPod for Serverless GPU Inference

Modal and RunPod offer leading serverless GPU platforms, but actual cost is driven by billing mechanics like idle timeouts and cold starts, not just the per-hour rate. This comparison breaks down deployment lock-in, serverless premiums, and strict EU compliance options.

Caspar Lehmkühler · 16 min read

Lower inference cost · Price comparisons

AWS Bedrock Pricing Explained: What You Actually Pay Per Token

AWS Bedrock token prices are only the baseline. To forecast your real inference costs, you must account for separate input and output rates, provisioned throughput commitments, and hidden data transfer fees, and compare those against EU-sovereign open-model endpoints.

Caspar Lehmkühler · 10 min read

Lower inference cost · Price comparisons

Azure OpenAI Token Pricing vs EU Open-Model APIs

Azure OpenAI's complex token pricing and PTU commitments can quickly inflate inference costs, and varying deployment types obscure true data residency. Moving to an EU-sovereign, open-model API drastically cuts total compute spend while guaranteeing GDPR compliance by design.

Maximilian Niroomand · 9 min read

Lower inference cost · Token & cache savings

Batch vs Real-Time Inference Pricing: When the Discount Wins

Major AI providers cut inference costs by 50 percent when teams route requests through asynchronous batch queues instead of real-time endpoints. Slashing spend requires isolating workloads that tolerate 24-hour turnaround times from those requiring interactive responses.

Caspar Lehmkühler · 8 min read

Lower inference cost · Price comparisons

EU-Hosted Inference Cost: The Sovereignty Premium Measured

The assumption that EU data sovereignty carries a pricing premium ignores the hidden costs of public cloud infrastructure. When accounting for hyperscaler egress fees, idle GPU waste, and the legal overhead of Schrems II compliance, EU-hosted inference is frequently cheaper.

Caspar Lehmkühler · 10 min read

Avoid vendor lock-in · Provider alternatives

Groq Alternatives in Europe: Fast Inference Inside the EU

While Groq's custom LPUs deliver massive token generation speed, European teams face severe transatlantic network latency that undermines these gains. By hosting models locally on sovereign infrastructure, enterprises recover the Time to First Token gap and ensure GDPR compliance.

Maximilian Niroomand · 11 min read

Switch models safely · Task fit

DeepSeek V4 Flash: 1M-Token Context for AI Products

DeepSeek V4 Flash introduces a 284B parameter MoE architecture with 13B active parameters, delivering low time-to-first-token latency and a 1,048,576-token context window. For AI-native products, this means high-throughput agent loops and long-context retrieval hosted natively in Europe

Magnus Grünewald · 12 min read

Model library · Text

Kimi K3 API: pricing, where to run it and 1M-context costs

Call Kimi K3 through the Lyceum OpenAI-compatible API as moonshotai/kimi-k3, EU-hosted, for $3.00 per million input tokens and $15.00 output. Its context window (the text the model reads in one request) holds 1M tokens, exactly 1,048,576. A million fresh input tokens cost $3.00, or $0.75 when we reuse an identical prompt start from memory.

Magnus Grünewald · 9 min read

Model library

DeepSeek-V4-Pro: specs, benchmarks, and how to run it on Lyceum

DeepSeek-V4-Pro is DeepSeek's largest open V4 model, and one request can hold up to 1M tokens. We serve its August 2026 release as deepseek/deepseek-v4-pro-0813 from the EU, for $2.00 per million input tokens and $4.00 per million output tokens. You call it with your OpenAI SDK, pay per token, and your prompts are processed in Paris or Finland. We don't keep them.

Maximilian Niroomand

Meet EU requirements · Transfer risk

Schrems II and LLM Hosting: Navigating Data Residency Risks

The legal landscape for AI infrastructure in Europe has shifted from theoretical concern to operational risk. The intersection of the GDPR, the US Cloud Act, and the phased implementation of the EU AI Act has created a complex environment for CTOs and ML engineers. While many US-headquartered providers offer 'EU Regions,' the underlying ownership of the infrastructure remains a critical point of failure for compliance. For startups handling sensitive medical, financial, or manufacturing data, the physical location of a GPU is only half the battle. The real challenge lies in jurisdictional sovereignty and the technical reality of how prompt data, model weights, and logs are managed across borders.

Justus Amen · 17 min read

Model library

GLM-5.2: specs, benchmarks, and how to run it on Lyceum

GLM-5.2 is Z.ai's open model for long coding tasks, and one request can hold up to 1 million tokens. We serve it as z-ai/glm-5.2 from European data centres for $1.50 per million input tokens, $0.38 for repeated input and $4.50 for output. You call it with your OpenAI SDK, pay per token with no base fee, and prompts are never stored.

Caspar Lehmkühler · 8 min read

Model library

FLUX.2 Klein: specs, benchmarks, and how to run it on Lyceum

FLUX.2 Klein is a small, fast image model from Black Forest Labs that turns a text prompt into an image in four steps. On Lyceum you call it through our image API with a prompt and an aspect ratio, and you get an image URL back. Check the licence before you build a product on it: the checkpoints carry different terms, and some do not allow commercial use.

Justus Amen

Model library · Embeddings

Qwen3-Embedding-8B: specs, benchmarks, and how to run it on Lyceum

Qwen3-Embedding-8B is Qwen's open model that turns text into embeddings, lists of numbers that place similar meanings close together. We serve it EU-hosted as qwen/qwen3-embedding-8b for $0.02 per million input tokens (23 October 2026). You keep your OpenAI SDK, your texts are processed in Paris or Finland, and indexing 100 million tokens costs about $2.

Magnus Grünewald

Model library · Vision

MiniCPM-V 4.5: specs, benchmarks, and how to run it on Lyceum

MiniCPM-V 4.5 scores 77.0 on OpenCompass in an efficient 8B package. With its novel 3D-Resampler, it compresses video tokens by 96x, making long-video understanding highly cost-effective.

Magnus Grünewald · 8 min read

Model library · Image

Image Ultra: specs, benchmarks, and how to run it on Lyceum

Image Ultra delivers high-quality image generation in under one second. Designed for latency-sensitive applications, it offers a drop-in OpenAI-compatible API on EU-sovereign infrastructure.

Maximilian Niroomand · 8 min read

Model library · Text

gpt-oss-120b: specs, benchmarks, and how to run it on Lyceum

gpt-oss-120b brings OpenAI's reasoning capabilities to the open-source ecosystem. With 117B parameters and a sparse MoE architecture, it delivers o4-mini-level performance while fitting on a single 80GB GPU.

Caspar Lehmkühler · 7 min read

Model library · Text

Hermes-4-405B: specs, benchmarks, and how to run it on Lyceum

Hermes-4-405B introduces a hybrid reasoning mode that balances fast responses with deep, think-tag deliberation. Now available on Lyceum's European infrastructure, it delivers strong math and coding performance without the censorship of proprietary models.

Justus Amen · 8 min read

Model library · Image

FLUX.1 Dev: specs, benchmarks, and how to run it on Lyceum

FLUX.1 Dev brings strong prompt adherence and photorealism to open-weights image generation. Learn how to deploy this 12B parameter rectified flow transformer on Lyceum's EU-hosted infrastructure.

Caspar Lehmkühler · 8 min read

Meet EU requirements · AI Act

EU AI Act High Risk System Classification Guide

The EU AI Act introduces strict obligations for high risk AI systems, with penalties reaching 15 million euros. Engineering teams must understand classification rules and infrastructure requirements to avoid regulatory roadblocks.

Justus Amen · 16 min read

Meet EU requirements · AI Act

EU AI Act Prohibited AI Systems Checklist for Engineering Teams

The grace period for unacceptable risk AI systems ended on February 2, 2025. Engineering teams running models that breach the Article 5 prohibitions now face fines up to €35 million or 7% of global turnover, whichever is higher.

Magnus Grünewald · 16 min read

Meet EU requirements · AI Act

EU AI Act Foundation Model Obligations 2026: A Technical Guide

The grace period is ending. By August 2026, the European Commission will actively enforce compliance for foundation models, turning data residency and infrastructure choices into critical engineering constraints.

Caspar Lehmkühler · 14 min read

Meet EU requirements · AI Act

EU AI Act Conformity Assessment: The GPU Infrastructure Guide

The high-risk deadlines now fall on 2 December 2027 and 2 August 2028. Your conformity assessment will fail if your underlying GPU infrastructure cannot prove data sovereignty, logging traceability, and strict access controls.

Magnus Grünewald · 12 min read

Scale without limits · Latency & throughput

vLLM vs TensorRT-LLM: production benchmark & guide

Teams that host open models themselves can spend weeks comparing serving engines like vLLM, SGLang and TensorRT-LLM, and the published rankings flip with every release. What decides speed and cost most is how much of each prompt repeats, because repeated text doesn't have to be read again. So measure that share and test the engines on a recording of your own traffic. Or skip the choice: on Lyceum's API we run the engine for you in Paris and Finland, and repeated prompt text is billed at a lower price.

Justus Amen · 14 min read

Scale without limits · Latency & throughput

LLM Tokens Per Second Benchmark: Measured TTFT on Lyceum

This page publishes Lyceum's own measured throughput and first-token latency per model, with the test conditions beside every number, so a workload can be sized against a real measurement rather than a borrowed one.

Maximilian Niroomand · 8 min read

Scale without limits · Cold starts & autoscaling

Serverless GPU Cold Start Latency: Architecture Comparison

Scale-to-zero GPU infrastructure promises massive cost savings, but a 40-second cold start will kill any real-time AI application. Here is a technical breakdown of where the time actually goes and how modern inference stacks are solving the VRAM bottleneck.

Caspar Lehmkühler · 14 min read

Scale without limits · Reliability & support

The 2026 guide to AI inference SLAs: uptime, economics and EU compliance

An inference SLA (service level agreement) is a contract promise about availability, with a penalty if it is missed. A guarantee needs GPUs reserved for your traffic, so pay-per-token APIs on shared GPUs rarely carry one. Our pay-per-token API has no SLA, and we agree one per contract when we reserve GPUs for you.

Justus Amen · 14 min read

Scale without limits · Latency & throughput

2026 LLM Inference Latency in Europe: GPU Cost Guide

Inference now accounts for the majority of AI GPU spend. Here is how European engineering teams are optimizing latency, throughput, and cost per token on H100 infrastructure in 2026.

Magnus Grünewald · 16 min read

Meet EU requirements · Data residency

EU vs US Inference API Latency: The Cost of Transatlantic AI

Sending inference requests across the Atlantic adds roughly 75 to 160 milliseconds of unavoidable fiber latency. For modern compound AI systems, that delay multiplies exponentially, degrading user experience while exposing sensitive data to US jurisdictions.

Maximilian Niroomand · 14 min read

Switch models safely · Evaluation

Llama 3 vs Mistral vs Qwen: 2026 Model Selection Guide

Choosing the right open-weight model is only half the battle. See how Llama 3, Mistral, and Qwen compare on VRAM, quantization, and serving cost, and how to size the infrastructure behind them.

Caspar Lehmkühler · 15 min read

Lower inference cost · Price comparisons

Cost Per Million Tokens: The 2026 Provider Comparison Guide

Inference now consumes up to 80% of enterprise AI compute budgets. Discover the true cost per million tokens in 2026 and why renting from US-based API providers is destroying your unit economics.

Magnus Grünewald · 13 min read

Run models without infra work · Agents & workloads

Tool calling latency in LLM inference: production optimization

When your agent calls a tool, users can wait seconds before anything happens, even though the tool itself answers in milliseconds. The delay is the model: it often writes a long reasoning text first, reads a long list of tool descriptions, or gets the call wrong and has to try again. Fix it by switching reasoning off for simple calls, keeping tool descriptions short and at the start of the prompt, and forcing a valid call. On Lyceum, Qwen3.8 27B runs with or without reasoning, both with tool calling, and repeated prompt starts are cached automatically at a lower price.

Magnus Grünewald · 15 min read

Run models without infra work · Agents & workloads

RAG pipeline GPU infrastructure: the engineering guide

Many teams price a RAG pipeline as one big GPU job and buy too much hardware. Only one of its four steps, writing the answer, needs large GPUs, and it drives almost all of the cost. Search can run on ordinary CPU servers, and the cheapest saving is to send fewer retrieved passages to the model. On Lyceum you call both the embedding model and the answer model per token through one API in Paris and Finland, so you pay for questions answered, not for idle GPUs.

Caspar Lehmkühler · 13 min read

Run models without infra work · GPU efficiency

Long context inference: GPU requirements & VRAM guide

Teams that want a model to read whole contracts or codebases are often surprised by the GPU bill. Every token a model reads takes GPU memory, so long requests need far more memory than the model alone: one 128K-token request on Llama 3.1 70B adds 42.9 GB. Size the GPUs for how many long requests run at once, store that memory in 8-bit, and cap request length at what users really send. Past 128K tokens, call a 1M-token model per token on Lyceum's EU-hosted API instead of buying GPUs.

Magnus Grünewald · 14 min read

Run models without infra work · Agents & workloads

EU Compliant AI Agent Infrastructure: The 2026 Engineering Guide

EU customers ask two things before they buy an AI agent: where does each model call run, and what is kept? An agent makes many calls per task and resends its growing history each time, so cost and memory add up quickly. Run the model in Europe with a provider that stores no prompts, budget per step, and write your own log of every step, because the provider has none to give you. On Lyceum, models run in Paris and Finland, and we keep only a usage record with token counts, never your prompts or outputs.

Caspar Lehmkühler · 14 min read

Lower inference cost · Unit economics

Agent Inference Cost Optimization: Engineering the 2026 Stack

Agentic workflows multiply token consumption several times over compared to standard chat interfaces. We break down the engineering techniques and infrastructure decisions required to keep LLM inference costs viable at scale in 2026.

Magnus Grünewald · 14 min read

Switch models safely · Task fit

2026 Open-Source LLM Comparison: Benchmarks & Enterprise Deployment

Open-source models now match proprietary alternatives in reasoning and coding. For European engineering teams, the challenge has shifted from model selection to sovereign, GDPR-compliant deployment.

Maximilian Niroomand · 14 min read

Lower inference cost · Serverless vs dedicated

Open Source vs Closed API LLM Cost Comparison

API token prices have plummeted, but at scale, pay-as-you-go models still drain budgets. We work the arithmetic on where self-hosting open-source LLMs becomes cheaper than closed APIs, with every assumption shown.

Caspar Lehmkühler · 14 min read

Run models without infra work · Agents & workloads

Deploy Whisper Large v3 GPU API: VRAM, performance & EU hosting

If you need to transcribe a lot of audio, running Whisper Large v3 yourself costs about 4 cents per audio hour in our example. The usual mistake is to pick the GPU by memory: Whisper fits on almost any GPU, so what decides the cost is how much audio one GPU transcribes per hour. Measure that speed on your own audio, keep the GPU busy, and size from there. On Lyceum you can run Whisper on GPU machines in European data centres billed per second, or run a backlog as a job that stops when it is done.

Maximilian Niroomand · 14 min read

Keep GPU jobs running · Orchestration

Migrating GPU Workloads from Slurm to Kubernetes: A Practical Guide

Moving from Slurm to Kubernetes often means trading predictable batch scheduling for YAML complexity and silent hangs. Navigate the transition, maintain high GPU utilization, and build a unified AI infrastructure stack.

Justus Amen · 13 min read

Keep GPU jobs running · Orchestration

How to Run a Production ML Pipeline Without a DevOps Team

Managing your own GPU infrastructure is a massive engineering bottleneck. Learn how to decouple compute from operations and run end-to-end ML pipelines without hiring a dedicated DevOps team.

Caspar Lehmkühler · 15 min read

Keep GPU jobs running · Reliability

GPU Fault Tolerance in Distributed Training: A Technical Guide

Hardware failures are inevitable when scaling AI workloads across hundreds of GPUs. Learn how to implement robust fault tolerance in distributed training to prevent catastrophic job restarts and wasted compute.

Magnus Grünewald · 14 min read

Keep GPU jobs running · Orchestration

GPU Cloud API CI/CD Automation: Scaling ML Pipelines

Managing GPU infrastructure manually slows down model deployment and inflates costs. Integrating GPU cloud APIs directly into your CI/CD pipeline enables automated testing, faster iteration, and scale-to-zero efficiency.

Caspar Lehmkühler · 13 min read

Lower GPU cost · Rent vs own

Total Cost of Ownership for a GPU Cluster in 2026

Building an on-premise GPU cluster seems like a path to compute independence. But for most AI teams, the hidden costs of power, cooling, and idle time quickly turn a capital investment into a financial sinkhole.

Magnus Grünewald · 14 min read

Keep GPU jobs running · Distributed training

Multi-GPU Tensor Parallelism Setup: Configuration and Optimization Guide

A 70B model needs about 140GB in FP16 and does not fit on one 80GB GPU. Tensor parallelism splits weight matrices across devices, at the cost of four all-reduce collectives per transformer layer in a training step.

Caspar Lehmkühler · 14 min read

Lower GPU cost · Rent vs own

On-Prem vs Cloud GPU Breakeven: The 2026 Infrastructure Guide

Deciding between buying an 8x H100 server and renting cloud compute requires more than comparing list prices. We break down the utilization thresholds, power constraints, and compliance factors that dictate your total cost of ownership.

Justus Amen · 15 min read

Pick the right GPU · VRAM sizing

Mixture of Experts VRAM Requirements: A Practical Guide for ML Teams

Mixture of Experts (MoE) architectures promise massive intelligence at a fraction of the compute cost. But when moving from research to production, ML teams quickly discover the hidden bottleneck: MoE models are ruthlessly memory-bound.

Magnus Grünewald · 14 min read

Avoid vendor lock-in · Exit planning

Multi-Cloud GPU Strategy: How to Avoid AI Infrastructure Vendor Lock-In

A Parallels-commissioned survey reports that 94 percent of organizations are concerned about vendor lock-in. Architect an open-stack, multi-cloud GPU strategy that keeps your AI workloads portable and cost-effective.

Maximilian Niroomand · 14 min read

Lower inference cost · Serverless vs dedicated

Inference Cost Per Token vs. Dedicated GPU: 2026 Economics

Token-based billing is a retail markup on compute. As your AI product scales, paying a US-based provider for every word generated becomes your largest line item. We break down the engineering math behind the switch to dedicated GPUs.

Caspar Lehmkühler · 16 min read

Keep GPU jobs running · Fine-tuning & RL

LoRA vs Full Fine-Tuning Memory Cost: VRAM Math

You have a 24GB GPU and an 8B model. The math says it should fit, but your training script crashes with an OOM error before the first epoch. We break down the exact VRAM requirements for full fine-tuning versus LoRA.

Justus Amen · 15 min read

Lower GPU cost · Idle waste

GPU Idle Cost Waste Calculator: Stop Paying for Idle Silicon

Idle GPUs can consume budget between experiments, while waiting for input data or during quiet inference periods. Estimate the cost allocated to unused capacity, then identify which part is actually avoidable under your billing agreement. Profiling, right-sizing and resource lifecycle controls help distinguish useful work, necessary headroom and preventable waste.

Maximilian Niroomand · 15 min read

Keep GPU jobs running · Fine-tuning & RL

FP8 Training on H100: Benchmarks and Memory Savings

Training a 70-billion parameter model in BF16 requires hundreds of gigabytes of GPU memory. Shifting to FP8 precision on NVIDIA H100s halves the bytes per element for the tensors actually held in FP8, master weights and optimizer states stay in higher precision, and NVIDIA's NeMo measurements show 1.30x throughput on Llama 3 8B and 1.43x on Llama 3 70B versus BF16.

Caspar Lehmkühler · 13 min read

Run models without infra work · GPU efficiency

GGUF vs GPTQ vs AWQ: the definitive LLM quantization framework

When you run an open model yourself, you have to pick a file: the original, or a smaller GGUF, GPTQ, AWQ or FP8 version. The wrong pick means the model won't load, or leaves too little GPU memory for your users. The rule is simple: GGUF for laptops, CPUs and Macs, AWQ or GPTQ for serving users on GPUs, and FP8 on newer GPUs such as the H100. On Lyceum you can deploy AWQ and GPTQ files on dedicated endpoints in Europe, or skip the choice and call the model per token on our API.

Justus Amen · 13 min read

Secure GPU capacity · EU GPU providers

The European AI Infrastructure Stack in 2026: A Technical Guide

The era of experimental credit-burning is over. With the EU AI Act enforcement deadline approaching, ML teams need infrastructure that delivers raw performance without compromising data sovereignty.

Maximilian Niroomand · 14 min read

Model library

Deploy DeepSeek R1 on European GPU cloud: VRAM, costs and compliance

To deploy DeepSeek R1 in full, plan for eight H200 GPUs, because its weights alone take about 690 GB. Its distilled 32B version, a smaller model trained on R1's answers, runs on one or two GPUs. You can run either on our GPU virtual machines in Europe, or call our per-token DeepSeek V4 models instead.

Magnus Grünewald

Run models without infra work

Deploy a Hugging Face model inference API: 2026 production guide

To deploy a Hugging Face model as an inference API, first work out its GPU memory: the weights plus what each running request holds. Then serve it with vLLM, an open-source serving engine that answers in the OpenAI API format, and require an API key on every request. Put steady traffic on reserved GPUs and bursty traffic on capacity billed per token.

Caspar Lehmkühler · 15 min read

Model library

Deploy Qwen 2.5 72B, Gemma 3, Phi-4: VRAM Sizing and vLLM Setup

This page is for engineers who run Qwen 2.5 72B, Gemma 3 27B or Phi-4 on their own GPUs. A GPU must hold the model's weights, the numbers it learned, plus a record of every token it has read. Ten 16,000-token requests need 198 GB for Qwen (2 x H200), 136 GB for Gemma (1 x B200) and 62 GB for Phi-4 (1 x H100).

Magnus Grünewald

Lower GPU cost

GPU cloud per-second billing comparison: stop paying for idle compute

Per-second billing charges a GPU machine only for the seconds it runs, so a 14-minute job costs 14 minutes of GPU time. Billing by the started hour charges the same job for a full 60 minutes. Neither rule helps when the machine keeps running after the job, because a running machine is billed whether or not it computes.

Justus Amen · 16 min read

Keep GPU jobs running · Distributed training

Multi GPU Distributed Training Setup Guide: Frameworks & Infrastructure

Scaling from a single GPU to a multi-node cluster introduces complex communication bottlenecks and fatal memory errors. Learn how to configure DDP, FSDP, and DeepSpeed while optimizing your infrastructure for maximum throughput.

Caspar Lehmkühler · 13 min read

Model library

Multimodal AI inference on European GPUs: compliance and cost optimization

Several of our open models read images you send through our OpenAI-compatible API, while others reject the image or quietly ignore it. An image is billed as input tokens, so its pixel size sets its price. We process image requests in Paris and Finland, don't keep them and never train on them.

Magnus Grünewald

Lower GPU cost · Billing models

Reserved vs On-Demand GPU Strategy 2026: The Engineer's Guide

Most AI teams over-provision GPU capacity out of FOMO, and much of what they pay for sits idle. Learn to architect a compute strategy that cuts costs without sacrificing performance.

Justus Amen · 15 min read

Model library

Run Vision Language Models on GPU Cloud: VRAM & Setup Guide

This page is for engineers who pick a GPU for a vision language model (VLM), a model that answers questions about images. Each image it reads costs GPU memory (VRAM) on top of the model itself, so cap the image size before you choose a GPU.

Justus Amen · 16 min read

Pick the right GPU · GPU head-to-head

NVIDIA H200 vs H100 Cost Performance Comparison

The NVIDIA H200 offers 76% more memory than the H100, but identical compute power. Discover exactly when the H200's higher hourly rate is justified for your AI infrastructure.

Magnus Grünewald · 13 min read

Lower inference cost · Serverless vs dedicated

LLM Inference Cost Per Token: Serverless vs. Dedicated Comparison

Inference cost per unit of model quality keeps falling, yet AI infrastructure bills continue to climb. We break down where dedicated GPUs become cheaper than serverless APIs, and how to work out your own threshold.

Maximilian Niroomand · 14 min read

Pick the right GPU · VRAM sizing

GPU Selection Guide: Inference vs. Training Workloads in 2026

Selecting the wrong GPU architecture inflates your cost per token or bottlenecks your training runs. Understanding the structural differences between inference and training workloads is the only way to right-size your infrastructure.

Caspar Lehmkühler · 14 min read

Keep GPU jobs running · Orchestration

The ML Engineer Guide to GPU VM SSH Access and Scaling

Managing local hardware creates bottlenecks, but legacy cloud pricing destroys budgets. You need raw, reliable GPU access that scales without locking you into proprietary ecosystems.

Justus Amen · 15 min read

Lower GPU cost · Billing models

GPU Per Second Billing: Cost Savings for AI Infrastructure

Hyperscaler billing models force AI teams to pay for idle time. Discover how per-second billing and scale-to-zero infrastructure can drastically reduce your GPU costs.

Magnus Grünewald · 13 min read

Secure GPU capacity · Provider head-to-head

GPU Provisioning Speed Comparison 2026: Benchmarks & Architecture

Waiting minutes for a cloud GPU instance to spin up is no longer acceptable for production AI. We break down the published 2026 provisioning data, the architectural differences driving them, and how to eliminate cold start bottlenecks.

Maximilian Niroomand · 14 min read

Secure GPU capacity · Provider trust

GPU Cloud SLA Uptime Comparison 2026: The True Cost of Downtime

Two hours of downtime on a 128-GPU H100 cluster wastes about 700 USD of compute at Lyceum's listed on-demand rate, before idle engineering time. Evaluate GPU cloud SLAs on exclusions, capacity and data residency, not on the headline number.

Caspar Lehmkühler · 13 min read

Lower GPU cost · Egress fees

Egress Fees: The Hidden Cost of GPU Cloud Infrastructure

You provisioned an H100 cluster based on the hourly rate. Then the invoice arrived, and data transfer charges had overtaken your compute estimate. Here is how to model the true cost of AI infrastructure.

Maximilian Niroomand · 14 min read

Pick the right GPU · GPU head-to-head

NVIDIA B200 vs H100 Inference Performance Benchmarks

Inference now dominates AI compute spend. If you are serving 70B+ parameter models, the architectural leap from Hopper to Blackwell fundamentally changes your unit economics.

Caspar Lehmkühler · 14 min read

Pick the right GPU · VRAM sizing

Best GPU for LLM Fine-Tuning in 2026: Benchmarks & VRAM Math

Stop guessing your VRAM requirements. We break down the exact math, real-world benchmarks, and infrastructure economics for fine-tuning LLMs on NVIDIA B200, H100, A100, and L40S GPUs.

Justus Amen · 13 min read

Secure GPU capacity · Reservations

Scaling GPU Infrastructure from Series A to Series B

Transitioning from Series A to Series B means moving from subsidized cloud credits to real unit economics. Learn to scale your GPU infrastructure efficiently while maintaining strict GDPR compliance and avoiding vendor lock-in.

Magnus Grünewald · 14 min read

Meet EU requirements · Transfer risk

US-Based Inference APIs vs. EU Sovereign Providers: A Strategic Guide

When hyperscaler credits expire, infrastructure decisions shift from prototyping speed to production sustainability. Here is why relying on US-based APIs introduces severe compliance risks, and how the open-source stack has closed the performance gap.

Maximilian Niroomand · 14 min read

Secure GPU capacity · EU GPU providers

Modal Alternatives: Serverless Python GPU Cloud in Europe

Proprietary serverless platforms offer excellent developer experience at a steep premium. For European AI teams, the hidden costs of vendor lock-in and cross-border data transfers require a shift to sovereign infrastructure.

Caspar Lehmkühler · 14 min read

Secure GPU capacity · EU GPU providers

RunPod Alternatives for EU Data Residency: The 2026 Engineering Guide

With key EU AI Act obligations applying from August 2026 and cumulative GDPR fines past €6.3 billion, European ML teams are re-examining US-based GPU marketplaces. Here is the technical framework for evaluating sovereign alternatives.

Justus Amen · 16 min read

Secure GPU capacity · EU GPU providers

US GPU Cloud Alternatives: The EU-Sovereign Guide for AI Teams

Relying on US-based budget GPU clouds exposes European AI teams to severe GDPR risks and capacity bottlenecks. Discover why transitioning to EU-sovereign infrastructure solves both compliance and cost overruns.

Magnus Grünewald · 13 min read

Lower GPU cost · Hyperscaler exit

Migrate ML Workloads from Legacy Clouds to an EU GPU Cloud

Hyperscaler credits expiring? Facing constrained GPU capacity and high egress fees? AI startups are moving to sovereign European infrastructure to regain control over costs and compliance.

Maximilian Niroomand · 14 min read

Secure GPU capacity · Provider head-to-head

Hyperstack vs European GPU Providers: The 2026 Infrastructure Guide

Global GPU clouds often force European AI teams into a difficult compromise: accept US-based data residency or pay hyperscaler premiums. For teams scaling inference and training, sovereign European infrastructure offers a structural advantage in both compliance and cost.

Justus Amen · 14 min read

Secure GPU capacity · EU GPU providers

GPU Cloud for Seed Stage AI Startups: 2026 Infrastructure Guide

Seed stage AI startups can spend a large share of their funding directly on compute infrastructure. Choosing the right GPU cloud determines whether you scale efficiently or burn through your runway before finding product-market fit.

Magnus Grünewald · 14 min read

Keep GPU jobs running · Orchestration

First GPU Cloud Setup: The ML Startup Guide to Infrastructure

Transitioning from local hardware or expiring cloud credits to production infrastructure is a critical inflection point for ML startups. This guide breaks down how to architect your first scalable, EU-sovereign GPU cloud environment without falling into vendor lock-in.

Caspar Lehmkühler · 13 min read

Lower GPU cost · Hyperscaler exit

Hyperscaler GPU Alternatives in Europe: The Infrastructure Guide

Expiring cloud credits and chronically underused GPU capacity are breaking unit economics for AI startups. Engineering leaders are migrating to specialized European infrastructure to cut costs and guarantee GDPR compliance.

Justus Amen · 13 min read

Avoid vendor lock-in · Provider alternatives

Fireworks and Baseten Alternatives in Europe: A Strategic Guide

US-based managed inference platforms offer excellent developer experiences but fail on EU data sovereignty and cost at scale. Learn how European ML teams are migrating to sovereign infrastructure to maintain compliance and reduce GPU spend.

Maximilian Niroomand · 13 min read

Lower GPU cost · Hyperscaler exit

Azure GPU Pricing Alternatives 2026

The initial wave of hyperscaler credits has dried up. Discover how AI startups are cutting compute costs while maintaining strict EU data sovereignty.

Justus Amen · 13 min read

Meet EU requirements · Certifications

ISO 27001 AI Infrastructure Certification Guide (2026)

Enterprise clients will not hand over proprietary data without proof of security. For AI startups, ISO 27001 certification is the baseline requirement to move from pilot to production.

Magnus Grünewald · 15 min read

Secure GPU capacity · Availability

EU GPU Availability 2026: Navigating the B200 & H200 Compute Crunch

The 2026 GPU shortage is a structural memory crisis, and NVIDIA itself describes cloud GPUs as sold out. European AI teams are securing B200 and H200 compute by bypassing traditional waitlists.

Caspar Lehmkühler · 15 min read

Secure GPU capacity · EU GPU providers

GPU Cloud Europe: The 2026 AI Startup Infrastructure Landscape

European AI startups are hitting the hyperscaler credit cliff right as the EU AI Act enforcement deadline approaches. Surviving 2026 requires moving from rented, US-based infrastructure to owned, EU-sovereign GPU clouds.

Justus Amen · 14 min read

Meet EU requirements · Data residency

Sovereign AI Infrastructure in Germany: A 2026 Guide

With the EU AI Act generally applicable since 2 August 2026, European AI teams are moving beyond hyperscaler credits toward sovereign infrastructure. This guide examines the technical and regulatory requirements for building compliant, cost-effective GPU stacks in Germany.

Magnus Grünewald · 15 min read

Meet EU requirements · Transfer risk

GPU Cloud Data Sovereignty: Navigating US and EU Infrastructure

As hyperscaler credits expire, AI startups face a critical choice between US-based convenience and European legal certainty. Understanding the jurisdictional reach of the US Cloud Act, and the fact that the EU AI Act itself imposes no data-residency requirement, is now a technical and operational necessity.

Maximilian Niroomand · 14 min read

Meet EU requirements · Transfer risk

Host LLM in Europe Without US Data Transfer: A Technical Guide

European AI teams face a critical choice: scale on US-based infrastructure and risk regulatory non-compliance, or build on sovereign EU foundations. This guide explores how to deploy high-performance LLMs in European data centres, and where the exceptions to that footprint actually sit.

Caspar Lehmkühler · 14 min read

Meet EU requirements · GDPR & contracts

GDPR AI Training Data Processing: A Technical Compliance Guide

As the EU AI Act's high-risk obligations are deferred to 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems, the intersection of data privacy and model training has moved from a legal gray area to a critical infrastructure requirement. For AI startups, staying compliant now requires more than just a DPA - it demands a fundamental shift in how training data is sourced, stored, and processed on European soil.

Magnus Grünewald · 15 min read

Meet EU requirements · Data residency

GDPR Compliant LLM Inference: A Guide for European AI Teams

European AI startups face a critical choice between high-performance inference and the data residency terms customers and regulators expect. As hyperscaler credits expire and scrutiny intensifies, teams must move to infrastructure whose processing locations and transfer mechanisms they can document, without giving up low latency.

Maximilian Niroomand · 15 min read

Meet EU requirements · Provider vetting

European Alternatives to US Inference APIs: A Sovereignty Guide

For European AI teams, the choice of inference infrastructure is no longer just about latency or price. Regulatory pressure and the high cost of US hyperscalers are driving a migration toward sovereign European alternatives that offer provable data residency.

Caspar Lehmkühler · 16 min read

Secure GPU capacity · EU GPU providers

European GPU Cloud Comparison 2026: Sovereignty and Performance

As hyperscaler credits expire and the EU AI Act deadline approaches, European AI teams are re-evaluating their infrastructure. This comparison breaks down the technical and economic trade-offs between US-hosted platforms and sovereign European GPU providers.

Justus Amen · 15 min read

Meet EU requirements · AI Act

EU AI Act Infrastructure Requirements: Deadlines and Duties After the AI Omnibus

European AI teams face a critical regulatory shift. While the initial bans on prohibited practices took effect on 2 February 2025, 2 August 2026 is the date the Regulation applies in general and the date the Commission gains its power to fine general-purpose AI model providers under Article 101. The AI Omnibus, in force since 27 July 2026, then moved the Chapter III obligations for Annex III high-risk systems to 2 December 2027, and high-risk systems captured by Article 6(1), AI systems that are, or are safety components of, products covered by the EU product legislation listed in Annex I, to 2 August 2028. For teams building in sectors like healthcare, critical infrastructure, or employment, the Act requires evidence about the AI system and its operation. The necessary controls depend on the system and the provider's or deployer's role, rather than on a particular cloud architecture. Initial compliance work for a single high-risk system is a material cost line, and ongoing monitoring adds operational overhead on top of it. Moving beyond the 'move fast and break things' era, engineering teams must now treat compliance as a core component of their technical stack.

Magnus Grünewald · 17 min read

Meet EU requirements · Certifications

C5 Certification for GPU Cloud: Navigating German AI Compliance

For AI teams in Germany, the transition from hyperscaler credits to production infrastructure often hits a regulatory wall. As the EU AI Act approaches its 2026 enforcement deadlines, BSI C5 has moved from a niche requirement to a standing procurement question, though it is mandatory in fewer places than assumed.

Caspar Lehmkühler · 15 min read

Meet EU requirements · Data residency

Data Residency for LLM APIs: A Guide for European AI Teams

European AI startups face a critical choice: optimize for speed using US-based APIs or prioritize compliance to win enterprise contracts. This guide explores why data residency is no longer optional for teams scaling LLM applications in regulated markets.

Justus Amen · 14 min read

Scale without limits · Latency & throughput

vLLM Production Deployment Guide: Scaling Sovereign Inference

Moving LLMs from experimental notebooks to production-grade infrastructure requires more than just raw compute. This guide explores how to navigate memory fragmentation, optimize KV caches, and maintain GDPR compliance while scaling vLLM in 2026.

Maximilian Niroomand · 9 min read

Scale without limits · Latency & throughput

Reduce LLM Inference Latency on GPUs: A Technical Guide

High latency in LLM inference drives up compute costs and degrades user experience. This guide explores the hardware and software strategies required to minimize Time to First Token (TTFT) and maximize throughput on modern NVIDIA GPUs.

Magnus Grünewald · 5 min read

Scale without limits · Cold starts & autoscaling

The Economics of Scale to Zero: Slashing GPU Inference Costs in 2026

Running dedicated GPU instances for bursty inference workloads is the fastest way to burn through venture capital. Scale-to-zero orchestration allows teams to eliminate idle compute costs without sacrificing the performance required for production-grade AI.

Maximilian Niroomand · 6 min read

Switch models safely · Migration plan

OpenAI Compatible API Self Hosted: A Guide for EU AI Teams

Relying on proprietary US-based APIs creates significant risks for European AI teams, from GDPR non-compliance to unsustainable scaling costs. By adopting a self-hosted, OpenAI-compatible architecture, you can maintain full control over your data residency while moving to per-second and per-token pricing you can model directly against your own traffic.

Caspar Lehmkühler · 7 min read

Lower inference cost · Serverless vs dedicated

Pay Per Token vs Dedicated GPU Inference: The Break-Even Guide

As hyperscaler credits expire, AI startups face a critical infrastructure fork: continue paying per token or move to dedicated GPUs. This guide breaks down the utilization math, latency trade-offs, and sovereignty requirements for European engineering teams.

Justus Amen · 7 min read

Run models without infra work · GPU efficiency

Multi-model serving on single GPUs with vLLM and PagedAttention

Teams with several fine-tuned versions of one open model often give each its own GPU, and most of those GPUs sit idle most of the day. They can share one GPU instead. If all versions come from the same base model, load them as LoRA adapters, small add-on files, on one vLLM server, so the base model sits in memory only once. If they are different models, run one vLLM server per model and give each a fixed share of the GPU's memory. On Lyceum you can run this on your own GPU machine in European data centres, with an H100, H200 or B200.

Magnus Grünewald · 6 min read

Scale without limits · Latency & throughput

NVIDIA Dynamo: A Technical Guide to Inference Orchestration

The recent release of NVIDIA Dynamo has fundamentally shifted the landscape for AI infrastructure leads. By bridging the performance gap between open-source frameworks and proprietary engines, this orchestration layer allows teams to maintain full portability without sacrificing throughput.

Maximilian Niroomand · 8 min read

Run models without infra work · Custom deployments

Host fine-tuned model production APIs: a technical guide

Teams with several fine-tuned versions of one model often give each version its own GPU, and pay for five GPUs where one would do. The fix: load the base model once and serve each version as a small add-on file, a LoRA adapter. Five full copies of a 27-billion-parameter model need five 80 GB GPUs, while one base model with five adapters fits on one. On Lyceum, our fine-tuning (in early access) trains the adapter on GLM-5.3 Flash or Qwen3.8 27B in Europe and serves it per token, so only the model name in your code changes.

Caspar Lehmkühler · 7 min read

Run models without infra work · Endpoint choice

Self-hosted LLM API gateway guide: architecture and infrastructure

Once a second team ships an LLM feature, nobody can say who spent what, and every app keeps its own keys and retry code. A self-hosted LLM API gateway fixes that: one server of your own sits between your apps and every model backend. Your apps call one address, and the gateway holds the keys and enforces each app's budget, rate limit and retries. You can point it at Lyceum's OpenAI-compatible API in Europe, where API keys can carry their own spend limit as a second safety net.

Justus Amen · 7 min read

Run models without infra work · Custom deployments

Deploying private LLM endpoints on GPU cloud: a 2026 strategy

Some teams want a language model behind their own address, on GPUs reserved only for them. There are two ways to get one: deploy a Hugging Face model as a dedicated endpoint and let the provider run it, or run your own container on a rented GPU machine and run it yourself. Both accept OpenAI-style requests, so your code only changes its address and key. On Lyceum both run in European data centres, and dedicated endpoints pause after an idle hour, so you don't pay overnight.

Maximilian Niroomand · 6 min read

Run models without infra work · Custom deployments

Deploying Llama 3 inference APIs on sovereign GPU clouds

Teams that want Llama 3 as their own API often pick GPUs by whether the model fits, then find it answers only one or two requests at a time. The GPUs need memory for the model and for every request it handles at once. So size them by the memory left after the model: Llama 3.1 8B fits on one 80 GB GPU, while Llama 3.1 70B needs 140 GB for the model alone. On Lyceum you deploy Llama 3 on a dedicated endpoint from its Hugging Face name, on GPUs in Europe, and an endpoint idle for an hour stops billing.

Justus Amen · 7 min read

Scale without limits · Latency & throughput

Optimizing LLM Inference Throughput with Batching Strategies

Maximizing GPU utilization requires moving beyond simple request-level processing. This guide explores how continuous batching and PagedAttention solve the memory bandwidth bottleneck for production LLM serving.

Magnus Grünewald · 6 min read

Meet EU requirements

EU sovereign inference platform comparison: 2026 technical guide

An EU sovereign inference platform is a European-run service that runs AI models on GPUs in European data centres. Its documents show where each prompt is processed, who can see it and what is kept. Compare platforms on those documents rather than on the word "sovereign", because the open models and their prices are often the same.

Magnus Grünewald · 20 min read

Run models without infra work · Endpoint choice

Serverless GPU inference: architecture, economics, and compliance

Running your own GPU for a language model means paying for every hour, even when nobody uses it. Serverless GPU inference solves that: the model already runs on shared GPUs, you call it through an API and pay only for the text it reads and writes. A quiet night costs nothing, and there is no GPU to start, size or switch off. On Lyceum you call open models this way in Paris and Finland, and prompts and outputs are not stored or used for training.

Justus Amen · 1 min read

Scale without limits

Serverless inference cold start latency: shorten it or hide it

In our worked example, an internal assistant that pauses after an idle hour is billed for 50 hours a week instead of 168. The price is five slow first requests, one each weekday morning. A request scheduled before staff arrive removes even those.

Maximilian Niroomand · 8 min read

Pick the right GPU · VRAM sizing

NVIDIA B200 180GB VRAM Model Requirements: A Technical Guide

The NVIDIA B200 introduces 180GB of HBM3e memory and native FP4 precision, fundamentally changing how AI teams provision infrastructure. Understanding its exact memory requirements is critical to preventing out-of-memory errors and maximizing cluster utilization.

Maximilian Niroomand · 13 min read

Pick the right GPU · GPU head-to-head

NVIDIA B200 vs H200 GPU for Inference: Architecture & Benchmarks

Choosing between the NVIDIA B200 and H200 dictates your inference latency and Total Cost of Compute. Discover how Blackwell's dual-die architecture and native FP4 support compare to Hopper's refined HBM3e memory.

Maximilian Niroomand · 14 min read

Pick the right GPU · GPU head-to-head

H100 vs B200 GPU Cost Efficiency Comparison for AI Workloads

Choosing the right GPU architecture dictates both the speed of your AI development and the sustainability of your infrastructure budget. Understanding the exact cost efficiency differences between the H100 and B200 is critical for optimizing large-scale machine learning workloads.

Maximilian Niroomand · 11 min read

Secure GPU capacity · Availability

NVIDIA B200 Availability in Europe 2026: A Technical Guide

The NVIDIA B200 brings unprecedented compute power to European data centers in 2026. Discover how to overcome the GPU utilization problem, optimize PyTorch workloads, and ensure strict EU data sovereignty.

Maximilian Niroomand · 12 min read

Lower GPU cost · Hourly rates

NVIDIA B200 GPU Cloud Pricing 2026: True Costs & Architecture

The NVIDIA B200 delivers 180GB of HBM3e per GPU as shipped in the HGX and DGX B200, plus native FP4 support, fundamentally changing AI compute economics. But with cluster utilization chronically low across the industry, raw hourly pricing tells only a fraction of the story.

Maximilian Niroomand · 15 min read

Lower GPU cost · Expired credits

AWS Credits Expired: A Strategic Guide for AI Infrastructure

For many AI scaleups, the expiration of AWS Activate credits, or of Google for Startups and Microsoft for Startups credits, marks the end of the 'experimentation phase' and the beginning of the 'optimization phase.' During the credit period, efficiency is rarely a priority; engineers often overprovision A100s or H100s for simple tasks because the cost is abstracted away. However, once the first real invoice arrives, infrastructure shifts from a line item to a primary driver of Cost of Goods Sold (COGS). This transition, often called the cloud cliff, demands a rigorous technical audit of your stack. Moving forward requires more than just cost-cutting; it necessitates a sophisticated approach to GPU orchestration, hardware selection, and data residency to maintain competitive margins.

Magnus Grünewald · 13 min read

Lower GPU cost · Hourly rates

Navigating the AWS GPU Price Increase in 2026

As AWS adjusts its EC2 pricing for high-performance GPU instances in 2026, AI teams face a critical choice between absorbing massive overhead or optimizing their stack. Understanding the drivers behind these increases is essential for maintaining sustainable ML development and deployment cycles.

Justus Amen · 11 min read

Lower GPU cost · Hourly rates

AWS P5 H100 Pricing Per Hour 2026: A Technical Cost Analysis

As we move into 2026, the cost of NVIDIA H100 compute on AWS remains a critical line item for AI teams. Understanding the shift from on-demand premiums to workload-aware orchestration is essential for maintaining competitive margins in model training.

Justus Amen · 10 min read

Pick the right GPU · VRAM sizing

Best GPU for Llama 3 Fine-Tuning: A Technical Engineering Guide

Fine-tuning Llama 3 requires a precise balance of VRAM capacity and memory bandwidth to avoid the dreaded Out-of-Memory errors. This guide breaks down the hardware requirements for 8B and 70B models, focusing on cost-efficient scaling and sovereign infrastructure.

Caspar Lehmkühler · 11 min read

Lower GPU cost · Rent vs own

Colocation vs Cloud GPU for ML: An Engineering Guide

Choosing between owning hardware in a colocation facility and renting cloud GPUs is a trade-off between operational velocity and long-term cost efficiency. For modern ML teams, the decision hinges on utilization rates, data residency requirements, and the hidden tax of infrastructure management.

Justus Amen · 11 min read

Secure GPU capacity · Provider head-to-head

CoreWeave vs Lambda GPU Cloud: The ML Engineer’s Guide to GPU Clusters

As AI teams move past hyperscaler credits, the choice between specialized GPU providers like CoreWeave and Lambda becomes a critical architectural decision. This guide breaks down networking, orchestration, and the hidden costs of underutilization in the modern AI stack.

Justus Amen · 13 min read

Meet EU requirements

Data residency and GDPR compliance in AI training

Training an AI model on personal data needs a GDPR legal basis for each dataset and a working route for objections and erasure. Data residency, where the data and the GPUs physically sit, decides whether a copy leaves the European Economic Area (EEA) and becomes a transfer. A training run makes many copies, so that answer has to hold for each one.

Magnus Grünewald · 14 min read

Lower GPU cost · Rent vs own

Dedicated GPU vs Cloud Instance: The Engineer's Guide to AI Infrastructure

Choosing between dedicated hardware and virtualized cloud instances is a critical architectural decision for AI teams. This guide breaks down the technical trade-offs to help you optimize for throughput, compliance, and total cost of compute.

Caspar Lehmkühler · 10 min read

Lower GPU cost

Egress fees GPU cloud comparison: the hidden cost of AI

Two large US clouds both charge about $0.09 per GB for data leaving Europe, after 100 GB free each month. Exporting a 70B model's full checkpoint once a day costs about $2,185 a month at one of them, in its Frankfurt region. From 12 January 2027, the EU Data Act ends egress charges for switching providers, but not for everyday traffic.

Justus Amen · 15 min read

Meet EU requirements · Data residency

EU Data Residency AI News: The Rise of Sovereign GPU Infrastructure

As the EU AI Act enters its enforcement phase, the era of 'compliance-blind' AI development is ending. Discover how sovereign GPU infrastructure in European data centers is solving the data residency puzzle without sacrificing ML performance.

Magnus Grünewald · 12 min read

Secure GPU capacity

The rise of the Europe GPU cloud startup: sovereignty and scale

A Europe GPU cloud startup is a young provider, based in Europe, that rents out GPUs in European data centres. With one, the company you sign with and the machine your job runs on are both in Europe. A US cloud's European region moves only the machine.

Magnus Grünewald · 15 min read

Secure GPU capacity · EU GPU providers

Choosing a German GPU Cloud Provider: Hosting Versus Contracting

For AI teams in Europe, the shift from US hyperscalers to a German GPU cloud provider is driven by more than GDPR. It is about egress fees, data sovereignty, and chronically low GPU utilization. Check where a provider hosts, though: several run their capacity elsewhere in Europe.

Magnus Grünewald · 10 min read

Lower GPU cost · Egress fees

The Engineer's Guide to GPU Clouds with No Egress Fees

Egress fees are a quiet line item on an AI project's budget, and they create a financial barrier to data mobility. For ML teams moving terabytes of checkpoints and datasets, choosing a GPU cloud with no egress fees is a strategic necessity for maintaining cost-efficiency and operational flexibility.

Justus Amen · 10 min read

Pick the right GPU · VRAM sizing

GPU for 7B vs 70B Model: A Technical Infrastructure Guide

Choosing between 7B and 70B models is not just a performance decision, it is a fundamental shift in infrastructure requirements. This guide breaks down the hardware specifications, memory constraints, and orchestration strategies needed to deploy these models efficiently.

Caspar Lehmkühler · 12 min read

Pick the right GPU · VRAM sizing

GPU Memory Requirements for Transformer Models: A Technical Guide

Understanding the exact memory footprint of Transformer architectures is the difference between a successful deployment and a frustrating Out-of-Memory (OOM) error. We break down the math behind weights, activations, and optimizer states to help you size your GPU clusters accurately.

Caspar Lehmkühler · 11 min read

Pick the right GPU · GPU head-to-head

H100 80GB vs A100 80GB: Fine-Tuning Performance and TCC Analysis

Choosing between the NVIDIA H100 and A100 for fine-tuning involves more than comparing VRAM capacity. While both offer 80GB, the architectural shift to Hopper introduces the Transformer Engine and FP8 support, fundamentally altering the throughput and cost-efficiency of modern AI workloads.

Caspar Lehmkühler · 11 min read

Run models without infra work

KV cache memory calculation for LLMs: a technical guide

The KV cache (key-value cache) is the record a language model keeps in GPU memory of every token it has read in a request. To size it, multiply the cache per token from the model's config file by the tokens per request and the requests at once. Add the model weights, and the total must fit in your GPU's memory.

Maximilian Niroomand · 12 min read

Secure GPU capacity · Provider head-to-head

Lambda Labs vs RunPod vs Vast.ai: Choosing Your GPU Cloud

The era of the general-purpose hyperscaler is facing a challenge from specialized GPU cloud providers. While AWS, GCP, and Azure offer vast ecosystems, their GPU instances often come with high overhead, complex networking, and significant egress fees. This has led ML engineers toward specialized platforms like Lambda Labs, RunPod, and Vast.ai. Each of these providers addresses a different segment of the market, from enterprise-grade clusters to decentralized marketplaces. However, as teams scale beyond initial experimentation, they often encounter the utilization trap, where expensive hardware sits idle or under-indexed. Understanding the architectural differences between these providers is essential for optimizing the total cost of compute and ensuring long-term project viability. Lyceum publishes this article and competes in this market.

Justus Amen · 13 min read

Lower GPU cost · Hyperscaler exit

ML Training Without AWS: A Guide to Sovereign GPU Infrastructure

Hyperscalers often trap ML teams with high egress fees and complex orchestration that leads to chronically low GPU utilization. Transitioning to a sovereign GPU cloud allows for better resource efficiency, support for GDPR compliance, and a significant reduction in the total cost of compute.

Magnus Grünewald · 10 min read

Secure GPU capacity · Availability

Nvidia H100 Availability Europe: A Guide for AI Engineering Teams

Securing high-performance compute in Europe has evolved from a simple supply chain challenge into a complex strategic decision involving data residency and utilization efficiency. For engineering teams, the focus is shifting from merely finding H100s to optimizing how they are deployed within sovereign borders.

Justus Amen · 11 min read

Secure GPU capacity

Top RunPod alternatives in Europe for sovereign AI development

A RunPod alternative in Europe runs your work on GPUs in European data centres, under a company based in Europe. Calls to open models move with a new base URL, and your own containers move as images. Lyceums per-token API serves GLM, DeepSeek V4, Qwen3.8 and MiniMax M3, which Runpod's per-token API did not list on 21 October 2026.

Magnus Grünewald · 8 min read

Secure GPU capacity · EU GPU providers

Sovereign Cloud Providers 2026: The Shift to AI-Native Infrastructure

As data privacy regulations tighten and AI compute demands skyrocket, reliance on US-based hyperscalers has become a strategic liability for European enterprises. In 2026, sovereign cloud providers are offering the specialized hardware and legal compliance necessary to scale AI without compromise.

Magnus Grünewald · 11 min read

Lower GPU cost · Billing models

Spot Instance GPU ML Training: A Technical Guide for AI Teams

GPU clusters often suffer from an average utilization of just 40 percent, leading to massive waste in AI budgets. Spot instances offer a path to 90 percent cost reductions, provided you can handle the technical complexity of preemption and state management.

Justus Amen · 11 min read

Lower GPU cost · Hyperscaler exit

Switching from AWS to a European GPU Cloud: A Technical Guide

Many AI teams find themselves locked into AWS due to initial credits, only to face recurring egress fees and utilization waste later. Transitioning to a European GPU cloud like Lyceum offers higher utilization and European data centers in Paris and Finland, without the hyperscaler tax.

Magnus Grünewald · 11 min read

Pick the right GPU · VRAM sizing

Which GPU for Fine-Tuning 70B Models? A Technical Guide

Fine-tuning a 70B parameter model is the ultimate test for AI infrastructure. This guide breaks down the hardware requirements, from VRAM math to multi-GPU orchestration, ensuring you don't waste budget on underpowered or overprovisioned clusters.

Caspar Lehmkühler · 12 min read

Keep GPU jobs running · Distributed training

ZeRO-3 vs FSDP: A Deep Dive into Memory Efficiency for LLMs

Scaling large language models requires moving beyond standard data parallelism to overcome the memory wall. This technical guide compares DeepSpeed ZeRO-3 and PyTorch FSDP to help engineers optimize GPU utilization and eliminate out-of-memory errors.

Maximilian Niroomand · 10 min read

Lower GPU cost

Migrating from AWS to dedicated GPUs: a performance and cost guide

Migrating from AWS to dedicated GPUs means moving your training and inference to GPUs that another provider reserves for you. Do it in five steps: package the code, copy the data, test on one GPU, move training, then switch inference. Before you move any data, ask AWS to cancel its fee for data leaving AWS. It does so only if you move all your data, or all of one service, off AWS. Without that, moving 50 TB out of AWS's Stockholm region costs about $4,394 in our example.

Justus Amen · 8 min read

Lower GPU cost · Hyperscaler exit

Beyond the Big Three: Optimizing ML Training on Alternative Clouds

Legacy hyperscalers charge a premium for general-purpose infrastructure that often leaves GPUs idle and budgets drained. Moving to specialized ML infrastructure reduces egress fees and eliminates the DevOps tax while maximizing hardware efficiency for large-scale training runs.

Magnus Grünewald · 8 min read

Lower GPU cost · Hyperscaler exit

High-Performance Alternatives to AWS SageMaker for AI Teams

AWS SageMaker AI combines compute with managed development, training and deployment features. A cheaper alternative depends on which of those features your team uses and the engineering work needed to replace them. Compare matched hardware, region, purchasing term and utilization, then include storage, transfers, migration and operations. Specialized GPU clouds can be an alternative for containerized training or inference, while teams that rely on SageMaker Pipelines, data tooling or managed endpoints may value the integrated service. Lyceum publishes this article and competes in this market.

Magnus Grünewald · 9 min read

Meet EU requirements

EU data residency for AI and analytics: a five-point checklist

EU data residency means your data is stored and processed inside the EU or the European Economic Area (EEA). For AI and analytics work, that includes prompts, model outputs, logs and dashboards. It also covers who can open the data from outside the EU, because the GDPR treats that access as a transfer.

Magnus Grünewald · 9 min read

Meet EU requirements · GDPR & contracts

GDPR Compliant GPU Cloud Europe: Sovereign AI Infrastructure

Scaling AI models in Europe requires more than just raw compute; it demands a legal and technical architecture that respects data sovereignty. As US hyperscalers face increasing scrutiny under the CLOUD Act, European startups are shifting to sovereign GPU clouds to simplify transfer assessments and vendor security reviews without sacrificing the performance of H100 and B200 clusters.

Magnus Grünewald · 6 min read

Pick the right GPU

Hardware recommendations for LLM fine-tuning: the 2026 guide

Fine-tuning memory depends first on the method, then on model size. Training every weight of a 27-billion-parameter model holds about 432 GB, which takes several GPUs. LoRA trains only a small add-on and needs 64 GB, or 22 GB as QLoRA with a 4-bit model, so one card is enough.

Caspar Lehmkühler · 6 min read

Pick the right GPU · VRAM sizing

How Many GPUs for Model Training? A Practical Scaling Guide

Throwing more hardware at a model does not always lead to faster convergence. We break down the math behind GPU scaling to help you avoid over-provisioning and maximize training efficiency while maintaining data sovereignty.

Caspar Lehmkühler · 7 min read

Pick the right GPU · VRAM sizing

GPU Selection Guide for ML Training: 2026 Performance Benchmarks

Choosing the wrong GPU cluster doesn't just waste budget, it kills momentum through Out-of-Memory errors and scaling bottlenecks. This guide breaks down the 2026 hardware landscape to help you architect for efficiency and data sovereignty.

Caspar Lehmkühler · 9 min read

Pick the right GPU · GPU head-to-head

H100 vs A100 Cost Efficiency: A Technical Deep Dive

Stop looking at hourly rates and start measuring cost-per-checkpoint. We break down why the H100's architectural leaps make it the superior choice for modern AI workloads despite the higher price tag.

Caspar Lehmkühler · 8 min read

Pick the right GPU · GPU head-to-head

A100 vs H100 for LLM Inference: The Engineer’s Guide to Efficiency

Choosing between the NVIDIA A100 and H100 is no longer just a question of budget. For engineers building the next generation of AI applications, it is a choice between two fundamentally different architectural approaches to the transformer block. The A100 was the workhorse of the first LLM wave, but the H100 was built specifically to solve the bottlenecks that emerged during that era. At Lyceum, we see teams struggling with OOM errors and high latency because they are trying to force modern, high-parameter models onto older hardware without considering the total cost of inference. This guide breaks down the technical reality of these GPUs to help you optimize your deployment.

· 7 min read

Keep GPU jobs running · Orchestration

Optimize Slurm GPU Allocation for High Performance AI Workloads

GPU scarcity and high operational costs make inefficient scheduling a terminal risk for AI startups. We break down how to tune Slurm for maximum throughput while maintaining the data sovereignty your enterprise clients demand.

Caspar Lehmkühler · 7 min read

Pick the right GPU · VRAM sizing

How to Right Size GPU Instances for ML Workloads

Most engineering teams waste a significant share of their compute budget on over-provisioned GPUs or lose days of productivity to Out-of-Memory errors. Finding the balance between VRAM capacity and compute throughput is the difference between a successful deployment and a drained runway.

Caspar Lehmkühler · 8 min read

Lower GPU cost · Idle waste

Stopping the Bleed: The Hidden Cost of GPU Overprovisioning

The race for H100s has left many startups with massive cloud bills and idle silicon. If your team is reserving 8-GPU nodes for workloads that never come close to filling them, you are subsidizing the inefficiency of legacy cloud providers.

Justus Amen · 7 min read

Lower GPU cost · Hourly rates

The Cost Per Training Run Calculator: A Guide for ML Engineers

Most AI teams realize their cloud bill is unsustainable only after the training run finishes. We break down the physics of compute costs and why Model Flops Utilization (MFU) is the only metric that actually matters for your bottom line.

Justus Amen · 6 min read

Lower GPU cost · Rent vs own

GPU ROI: Beyond the Hourly Rate in ML Infrastructure

Most ML teams focus on the hourly cost of an H100 while ignoring the idle time and DevOps friction that actually destroy their margins. True ROI requires a shift from measuring price-per-hour to measuring price-per-successful-training-run.

Justus Amen · 6 min read

Lower GPU cost · Idle waste

Strategies to Reduce GPU Cloud Costs for ML Training

GPU spend is often the single largest line item for AI teams today. We examine how to cut these costs materially through automated orchestration, strategic hardware selection, and sovereign cloud architectures.

Justus Amen · 8 min read

Lower GPU cost · Idle waste

GPU Utilization Too Low: How to Fix Compute Bottlenecks

When you monitor your training jobs and see GPU utilization sitting far below what the hardware can deliver, you are paying for capacity you never use. In high-performance environments, especially those utilizing NVIDIA H100 or A100 GPUs, the hardware is often faster than the software feeding it. This mismatch creates a 'starvation' effect where the GPU completes its work and waits for the next batch. It is common in teams whose data pipelines were built for smaller models and never updated for modern compute scales. Fixing this requires a systematic approach to profiling, data orchestration, and memory management to ensure your compute investment is fully leveraged.

Maximilian Niroomand · 9 min read

Keep GPU jobs running · Memory & OOM

PyTorch Memory Profiling in Production: A Guide to Efficiency

Out-of-memory errors in production are more than a technical hurdle; they represent a direct failure in system reliability and cost efficiency. Effective memory profiling requires a shift from local debugging to continuous, low-overhead monitoring that identifies leaks and fragmentation before they crash your sovereign GPU cluster.

Maximilian Niroomand · 7 min read

Keep GPU jobs running

CUDA out of memory with LLMs: where VRAM goes and how to fix it

A CUDA out of memory error means your job asked the GPU for more memory than the card has. When you fine-tune a large language model, most of that memory holds the optimizer's running statistics and saved intermediate results, not the model itself. Shrink the statistics and the saved results first, and add GPUs only when both are as small as they can get.

Maximilian Niroomand · 7 min read

Pick the right GPU · VRAM sizing

How to Predict VRAM Usage for PyTorch Models

The dreaded CUDA Out of Memory error is not a random occurrence but a predictable failure in resource planning. Understanding the exact byte-level requirements of your model allows you to optimize performance and maintain infrastructure independence.

Maximilian Niroomand · 5 min read

Pick the right GPU · VRAM sizing

GPU Memory Calculator for Deep Learning: A Technical Guide

Running out of memory mid-training is a costly engineering failure that stalls innovation. Understanding the precise breakdown of weights, gradients, and optimizer states is the only way to optimize your compute budget and avoid the dreaded CUDA Out of Memory error.

Maximilian Niroomand · 7 min read

Keep GPU jobs running · Memory & OOM

Solving OOM Errors in 70B Model Fine-Tuning

You hit the wall. Your terminal is flooded with CUDA Out of Memory errors while trying to fine-tune a 70B parameter model. This is not a hardware shortage; it is a memory orchestration challenge that requires a precise technical response.

Maximilian Niroomand · 6 min read

Keep GPU jobs running · Memory & OOM

Solving CUDA Out of Memory Errors in Llama Fine-Tuning

The torch.cuda.OutOfMemoryError is the most common roadblock for engineers fine-tuning Llama models. This guide breaks down the technical strategies to bypass VRAM limits and scale your training on sovereign infrastructure.

Maximilian Niroomand · 7 min read

Keep GPU jobs running · Memory & OOM

How to Prevent OOM Errors in PyTorch Training

Nothing halts a training run faster than the dreaded CUDA Out of Memory error. As models grow and datasets expand, managing VRAM becomes a critical engineering discipline rather than a trial and error exercise.

Maximilian Niroomand · 6 min read

Pick the right GPU · VRAM sizing

GPU Memory Estimation: A Guide to VRAM Requirements

Out-of-memory (OOM) errors are the silent killers of training productivity and budget. Learn how to mathematically predict your GPU memory footprint before you provision a single node on your cluster.

Maximilian Niroomand · 8 min read

Your next workload starts here