Migrate an LLM workload from a US inference provider to the EU

Why LLM workloads move from US providers to the EU

Most of the time in an EU migration goes into paperwork and testing. The code change is small. An LLM (large language model) workload is one feature of your product that sends text to a model and uses the answer.

How do we move an LLM workload from a US inference provider to the EU without rewriting the app? The question usually starts with a customer. A bank, an insurer or a public-sector buyer asks where its data is processed, and a US address slows the deal.

The legal background is the GDPR, the EU's data protection regulation. The EEA is the EU plus Iceland, Liechtenstein and Norway, and a third country is any country outside it. Article 44 allows a transfer of personal data to a third country only under Chapter V, the GDPR's chapter on such transfers. Article 45 allows it where the European Commission has decided the country is adequate. For the United States, that decision covers only companies certified under the EU-US Data Privacy Framework. So check whether your US provider is certified. Otherwise, Article 46 requires safeguards such as standard contractual clauses, the EU's model contract for transfers. This article explains the rules; it is not legal advice.

Quality is less of a reason to stay. Open-weight models are models whose weights anyone can download. Closed models keep their weights private. Epoch AI measures models on one aggregate capability score, the Epoch Capabilities Index. On it, the best open-weight models have trailed the leading closed ones by four months on average since January 2026. Its data was updated on 29 May 2026.

Lyceum is a sovereign European AI inference provider for open-source models. Our serverless models run in Paris and Finland. Prompts and outputs are processed, not stored, and never used for training. Some US per-token APIs differ. Together AI's documentation says it stores prompts and responses unless an organisation turns on zero data retention. Modal, a US inference provider, documents that requests to its shared pay-per-token endpoints route through us-west, a US region. We read both pages on 23 October 2026. Our guide to European alternatives to US inference APIs compares more options.

Inventory your LLM calls and the data they carry

Write down every place your code calls a model before you change anything. Each line of the inventory records five things:

Flag calls to an old text completions endpoint. Our serverless inference docs list no /completions endpoint: all text generation goes through /chat/completions with a list of messages.

The same list feeds your paperwork. Your record of processing activities (GDPR Article 30) lists the personal data you process and why. It names the recipients. That includes recipients in third countries and any transfer to one. If a call carries personal data, your US provider is in that record today. Our article on whether you need a DPIA, a data protection impact assessment, covers the risk check that can come with it.

Map each workload to an open model we serve

Pick one open model per workload, then price it with your own token counts. A token is a short piece of text, and you pay per token read and written. Our hypothetical example for the rest of this article is a tax and accounting tool that summarises client documents.

The prices come from our MiniMax M3 model page. GET /models on our API returns the model IDs we accept right now.

The hypothetical tool sends 100,000 requests a month. Each request carries 6,000 input tokens, and 4,000 of them are the same instructions every time. Each answer is 1,000 tokens, including the model's reasoning. Reasoning is text the model writes before its answer, and it counts as output.

The repeated instructions lower the bill. Prompt caching works like this: when requests start with the same text (a shared prefix), that prefix is billed at the cheaper cached-input price. Caching is best effort, so the calculation shows a best case and a worst case:

Calculated example, hypothetical inputs, MiniMax M3 list prices of 23 October 2026
new input:     2,000 x 100,000 = 200M tokens x $0.40 = $80.00
cached input:  4,000 x 100,000 = 400M tokens x $0.10 = $40.00
output:        1,000 x 100,000 = 100M tokens x $2.00 = $200.00
all 4,000 repeated tokens cached: $80.00 + $40.00 + $200.00 = $320.00 per month
no cache hits: 600M input tokens (6,000 x 100,000) x $0.40 = $240.00
               plus $200.00 output = $440.00 per month

The bill for this example lies between $320 and $440 a month. It does not show your old provider's bill, answer quality on your documents or latency. Our guide on what breaks when switching models covers quality.

Check tool calls before you commit. A tool call is the model asking your code to run a function you defined. A forced tool call is a request whose tool_choice names one function the model must call. In our checks on 17 September 2026, some models ignored it, while minimax/minimax-m3 honoured it. If your app depends on one, check message.tool_calls and fall back when it is empty.

Swap the base URL, API key and model ID

Our serverless endpoints are OpenAI-compatible. You point your OpenAI client at our base URL, the address every request goes to, and swap in our key and a model name. Streaming, tool calling and usage accounting work the same.

1. In the dashboard, open API Keys and press Create key. The full key is shown exactly once.
2. Optionally, an owner or admin of your organisation in our dashboard sets a spend limit on the key.
3. Set the base URL to https://api.lyceum.technology/openai/v1 and send the key, which starts with lk_, as a Bearer token.
4. Change the model name to the one you mapped.

In Python, the change sits in the client and one argument:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LYCEUM_API_KEY"],
    base_url="https://api.lyceum.technology/openai/v1",
)

resp = client.chat.completions.create(
    model="minimax/minimax-m3",
    messages=[
        {"role": "system", "content": "Summarise the client document in five bullet points."},
        {"role": "user", "content": "<document text>"},
    ],
)
print(resp.choices[0].message.content)

Keep the instructions that never change at the front, as in the system message above. That is the part the cache can reuse. A request may run for up to 300 seconds, so stream long outputs. Retry 429, 502, 503 and 504 with backoff, and treat other 4xx errors as permanent.

Update the DPA and your record of processing

The paperwork has two parts: a contract with the new provider and an update to your own record. A DPA (data processing agreement) sets how a provider handles personal data for you. GDPR Article 28 requires such a contract between processor and controller. The controller decides why data is processed, and the processor handles it on the controller's behalf.

Under our DPA, you are the controller and we are the processor. Our DPA is Annex 1 of our terms, so it applies when you accept them. There is no separate document to sign.

Your legal team will ask what we keep. A request leaves a usage record: your organisation and user, the model, token counts and the time. It never contains the prompt or the response. Our subprocessors, the companies that run parts of the service, are mainly in the EU and EEA. Anything outside is covered by GDPR safeguards such as standard contractual clauses. We tell you before we add or replace one, and you can object within 10 business days. Ask for the list on our legal and data handling page.

Then update your own record. Our advice is to add us as a recipient on the day traffic moves. Remove the old provider's transfer entry once it receives no more data.

Test on your own traffic, then cut over

Test the new endpoint on a copy or a slice of your real traffic before the cut-over. Compare answers, cost and errors with the old provider. For the document example, check the summaries and whether the bill lands in the $320 to $440 range.

Keep the old configuration as a fallback until the new path holds. Our public status page shows live status, per-model latency and past incidents. Serverless inference is pay as you go with no commitment, so one workload is a cheap first test. Our article on what switching providers costs lists the costs beyond the code.

Move the next workload once the first one holds. You can leave again the same way, by changing the same three settings. A steady, large workload can later move to dedicated inference, a model on GPUs reserved for you.

Create an API key and point one workload at our base URL, then compare it on your own traffic.