What is an EU-hosted LLM proxy?

An EU-hosted LLM proxy is a server-side layer that sits between your applications and every large language model provider you use. It intercepts each request, applies your organisation's data and access rules, forwards the request to a provider inside EU jurisdiction where possible, and keeps a record of what happened. All of that runs on infrastructure physically located in the European Union.

The category exists because European organisations are being asked to prove three things at once: where their prompts go, how their AI costs are attributed, and what records they can produce during an EU AI Act review. A proxy is the operational answer.

Why "EU-hosted" is different from "GDPR-friendly"

Any US-based AI provider will tell you they respect GDPR. The legal distinction that matters is not whether data is handled carefully, but where it physically resides while being processed and which government has legal access to it.

The Schrems II ruling from the Court of Justice of the European Union (Case C-311/18, 16 July 2020) struck down the EU-US Privacy Shield precisely because US intelligence services can compel disclosure of data held by US companies even when that data belongs to European customers. The EU-US Data Privacy Framework that followed has restored a legal basis for many transfers, but it has not removed the underlying tension.

The practical consequence for regulated European organisations is this. Routing prompts through a US-region endpoint is a transfer under Chapter V of GDPR, and it requires either a transfer mechanism or a lawful derogation. Routing prompts through an EU-hosted proxy that forwards to a provider's EU region avoids the transfer entirely for that request.

The three failure modes an EU-hosted proxy prevents

1. Silent data leakage across jurisdictions. A developer signs up for a US AI service on a company card. Prompts containing customer names, invoice numbers, and product roadmaps start flowing to US servers before the DPO has heard of the tool. An EU-hosted proxy centralises the endpoint so this cannot happen without the platform team seeing it.
2. Cost that finance cannot explain. Every LLM provider bills after the fact by token count. Without attribution at the request level, finance sees a single monthly total and has no way to allocate it to teams, products, or workflows. A proxy tags every request with the team and project it belongs to.
3. Audit evidence that does not exist. The EU AI Act (Regulation (EU) 2024/1689) requires providers and deployers of high-risk AI systems to keep records of automated events for the lifetime of the system under Article 12, maintain data governance documentation under Article 10, and demonstrate accuracy, robustness, and cybersecurity measures under Article 15. If those records were never generated at request time, they cannot be reconstructed later. A proxy generates them automatically.

What an EU-hosted LLM proxy does not do

Vendor marketing often blurs these boundaries. We name them explicitly.

• It is not an AI model. A proxy calls existing models on your behalf. It does not do inference itself. Model capabilities, latency, and accuracy come from the upstream provider.
• It does not certify EU AI Act compliance. The Act does not permit a vendor to certify a customer. A proxy produces records that the customer's own risk assessment and human oversight processes must then use. Compliance is a shared responsibility with a strict split.
• It does not replace legal review. A DPO still needs to sign off on which providers may be used for which data categories. A proxy makes the "which" question much easier to answer, but it does not answer it.

How Ciralgo implements the pattern

Ciralgo sits between your applications and every LLM provider your organisation uses. Applications continue to call the OpenAI or Anthropic SDK, only the base URL changes. Ciralgo then does five things per request.

1. Route to a provider inside the EU where the workload allows.
2. Redact personal data at the proxy layer before the request leaves your infrastructure.
3. Attribute the request to a team, project, and workflow using a header your app supplies.
4. Log a tamper-evident record of the request, the routing decision, and the response metadata.
5. Return the response using the same shape the calling SDK expects.

Full technical detail lives in the docs. The product overview walks through the wire-compatibility model. The documentation lists the concrete provider regions Ciralgo can currently reach and maps each Ciralgo record to the specific AI Act article it satisfies.

Providers Ciralgo can route to inside the EU

The table below is a snapshot. Provider EU availability changes as vendors open new regions and deprecate old ones. This page is refreshed quarterly.

Provider	Model family	EU region(s) available	Ciralgo default
OpenAI	GPT-4o, GPT-5	Ireland via Data Zones for eligible enterprise contracts	Route to Data Zone when the customer has the contract; otherwise fall back to the customer's chosen policy
Anthropic	Claude family	Frankfurt via AWS Bedrock	Route to Frankfurt for eligible enterprise workloads
Mistral AI	Mistral Large, Codestral	France	Default route
OVHcloud AI Endpoints	Llama, Mixtral, and other open weights	Gravelines, Strasbourg	Default route
Scaleway Generative APIs	Llama, Mistral	Paris	Default route where the workload fits
Cohere	Command R+	Not yet in EU region as of the last refresh	Blocked at policy level by default; can be enabled per customer
Google Gemini	Gemini 1.5, 2.0	Not offered as EU-only endpoint at the time of writing	Blocked at policy level by default

Providers where EU residency is not currently possible

Not every model has an EU-region endpoint. Where residency is a hard requirement, Ciralgo defaults to blocking those providers and lets you enable them per workload with an explicit sign-off. Common cases:

• Frontier models with reasoning modes that are only served from the provider's home region.
• Fine-tuned closed models hosted on the training region for latency reasons.
• Providers still building out their EU footprint.

The honest position is that "EU-hosted" today means EU-first with named exceptions your DPO has approved. It does not mean "no US endpoint under any circumstances." Any vendor claiming the latter is either misrepresenting their coverage or serving from a much smaller subset of models than you might need.

EU-hosted AI providers landscape in 2026

The EU-hosted AI conversation has grown beyond the proxy layer itself. European buyers searching for "EU hosted AI" in 2026 are asking three questions at once: which model providers run inference inside EU jurisdiction, which of them offer the specific model class the workload needs, and how a proxy layer connects them into one operational surface.

The EU-hosted AI provider landscape falls into three categories.

EU-native model providers. Mistral (France) and Aleph Alpha (Germany) with primary infrastructure and legal entity inside the EU. Strong on European-language work, sovereignty guarantees, and public-sector suitability. See the deeper vendor comparison on the Dutch-language EU-hosted AI-tools guide which covers each vendor's data-residency posture, model tiers and typical enterprise use case.

US-origin model providers with EU inference regions. OpenAI (EU data zone for ChatGPT Enterprise and Team, Azure OpenAI Service in Sweden and West Europe), Anthropic (Claude on AWS Frankfurt with contract holder in Ireland), Google (Gemini in Belgium and Frankfurt regions on Vertex AI). Data at rest and inference at runtime stay inside EU jurisdiction when configured correctly. CLOUD Act exposure remains at the corporate-parent level.

EU-hosted managed inference platforms. OVHcloud AI Endpoints, Scaleway Managed Inference, Nebius, and DeepInfra EU host open-weight models (Mistral, Llama, Phi, Gemma) with per-token pricing inside EU regions. Useful for teams that want SLM economics without running their own GPU cluster.

For buyers searching specifically for EU hosted AI models (rather than EU hosted AI platforms), the relevant question is which model families run inference inside EU jurisdiction at production quality. As of 2026: Mistral Large 2 and Codestral (France), Aleph Alpha Luminous (Germany), Llama 3.3 70B and Mixtral 8x22B (via OVHcloud, Scaleway, Nebius EU regions), Phi-3 and Gemma (via EU managed-inference platforms), Claude Sonnet and Haiku (via AWS Frankfurt Bedrock for eligible enterprise contracts), and GPT-4o and GPT-5 (via OpenAI EU Data Zone for eligible Enterprise and Team contracts, or via Azure OpenAI Sweden and West Europe regions). Anthropic Claude Opus and Google Gemini 2.5 Pro are not currently available as EU-only endpoints as of the last refresh; a proxy layer blocks them at policy level by default. The EU hosted AI models list changes multiple times per year as providers open new regions, so any deployment decision that depends on model-region availability should verify at the point of deployment rather than at the point of policy-writing.

A proxy layer like Ciralgo sits above all three categories, so the model choice per workflow is decoupled from the vendor lock-in typically forced by native SDKs. For the routing decision framework specifically, see small language models in 2026. For the tag-less attribution mechanics that make the multi-provider setup accountable, see AI spend attribution.

What to check before choosing an EU-hosted LLM proxy

1. Where does the proxy's own control plane run? A proxy that routes to EU providers but hosts its own control plane in the US has not solved the problem.
2. Where are the logs stored? Same question. Request logs contain the sensitive material, not just the routing decisions.
3. Which specific provider regions can the proxy reach? Ask for the current list, not a headline claim.
4. How does the proxy handle the case where the requested provider has no EU region? The right answer is "policy-driven block with per-workload override," not "silent US fallback."
5. What records are generated per request, and in what format? Article 12 of the AI Act specifies logs of automated events, and auditors read this literally. CSV export is not enough for a serious audit.
6. How does it handle personal data in prompts? Detection at the proxy layer is table stakes. Ask what pattern set is used and how false positives are handled.
7. What is the DPA, and what is the current sub-processor list? These are the two documents your procurement and DPO will read first.

Frequently asked questions

Is an EU-hosted LLM proxy the same as an AI gateway?

Overlapping but not identical. "AI gateway" usually refers to a US-market category focused on cost control, rate limiting, and provider fallback. "EU-hosted LLM proxy" adds data residency and EU AI Act evidence generation as first-class concerns.

Do we still need a DPA with each underlying provider?

Usually yes. The proxy is a data processor with respect to the customer, and the underlying provider is a sub-processor. Standard sub-processing terms in the proxy's DPA cover this pattern.

What about latency?

Modern EU-region endpoints have latency comparable to US regions for European users. The proxy layer adds a small overhead, typically under a hundred milliseconds.

Can we self-host the proxy?

Some deployment models allow this. Self-hosting is not the same as compliance, though. See the security page for how residency is handled without self-hosting for most customers.

Related reading

For teams that already use two or more AI providers, three articles cover the practical decisions the proxy is designed to make easier.

• How to route between OpenAI and Anthropic cheaply, with audit logs intact: the routing counterpart to this infrastructure page. Three routing strategies, the six audit-log fields that matter regardless of strategy, and why small multi-provider teams face the same problem as larger ones.
• LLM provider routing policy: how Ciralgo decides: the governance-level view of routing, aimed at the person signing the DPA or the AI Act conformity assessment.
• AI audit logging: a deeper look at the log fields the EU AI Act expects from every AI system.
• How to attribute AI costs to teams: the attribution use case the proxy enables at any team size.
• Small language models in 2026: when SLMs beat frontier LLMs: the model-choice lever the proxy makes routable per workflow.
• Fine-tuning under EU jurisdiction: cost, privacy and routing decisions: when to fine-tune, how to keep artifacts in EU jurisdiction, and how the routing layer treats fine-tuned models.
• AI energieverbruik en CSRD-rapportage voor Nederlandse organisaties: de rapportage-verplichting die de per-verzoek-logs uit de proxy tot een verdedigbaar carbon-cijfer maakt.
• Ciralgo product overview
• Documentation
• Security posture
• Data Processing Agreement
• Current sub-processors list

Last refreshed: 8 July 2026. This page is reviewed quarterly.