Provider routing governance

LLM provider routing policy is what turns a naive "pick the closest region and go" mindset into a defensible enterprise choice. Every time a request arrives at Ciralgo, four questions get answered before the request leaves the platform, and each answer generates evidence an auditor can read months later.

That framing sounds heavy for what teams usually treat as load balancing. It is heavy on purpose. Under Regulation (EU) 2024/1689 and under most enterprise data-protection agreements, the choice of which provider processes a prompt is a substantive processing decision. As a result, it requires a policy, a record, and a way to explain the outcome. This page describes how Ciralgo makes that decision, what shapes the LLM provider routing policy, and how the resulting records get used.

The audience for this page is the person who has to sign the DPA or the AI Act conformity assessment. If that person is not you, this page is what your platform team should be able to hand over.

The four inputs to every LLM provider routing policy decision

Ciralgo does not pick providers based on availability or cost alone. Every routing decision is a function of four inputs, evaluated in this order:

1. The organisation's LLM provider routing policy. This is the set of rules the customer has configured: allowed providers, jurisdictional constraints, sensitivity levels, budget caps, and workflow-specific overrides.
2. The request context. The model requested (gpt-4o, claude-sonnet-4-6, and so on), the attribution headers (X-Ciralgo-Team, X-Ciralgo-Project, X-Ciralgo-Workflow), any explicit region pin, and any override token.
3. The provider surface. Which providers are online, which regions of each provider are online, and which models each region currently serves.
4. The compliance state. Whether the request carries an active consent record if consent is required, whether the requesting workflow is under a pause order, and whether the retention class assigned to this request is enabled.

If any of the four inputs makes the request impossible to route legally, Ciralgo refuses the request with a specific error rather than silently degrading. This is a load-bearing design choice. Specifically, a silent fallback from an EU provider to a US provider is exactly the failure mode auditors look for.

Jurisdiction policies

The most common policy an organisation configures is jurisdictional. Ciralgo supports three shapes.

EU-only. The request must be handled by a provider region physically located inside the EU or the wider EEA. If no EU region can serve the requested model, the request is refused with PROVIDER_BLOCKED and a reason of jurisdiction. This is the default for most regulated deployments.

EU-first with named exceptions. The request goes to an EU region if one exists for the requested model. If not, Ciralgo checks the exception list. If the requested model has been added to the exception list by name (and the exception has an active timestamp), the request is forwarded to the exception provider. Every exception triggers a PROVIDER_ALLOWED_BY_OVERRIDE record naming the exception, the person who authorised it, and the expiry.

Region-pinned. The request must be handled by a specific provider region (e.g. eu-west-1 on AWS Bedrock, or europe-west3 on Google Cloud Vertex). This is useful when a customer contract or a downstream data flow requires evidence that data touched exactly one region and no other. Any request that cannot be routed to the pinned region is refused.

An organisation can combine shapes. It is common to see "EU-only for openai/gpt-4o, region-pinned to eu-west-1 for anthropic/claude-sonnet-4-6, EU-first with a US exception for openai/o1". The policy engine evaluates every rule against every request.

Cost caps and budget enforcement

Ciralgo enforces budget in three places, from tightest to loosest.

Per-request caps. The request carries a max_tokens (for OpenAI-shaped) or max_output_tokens (for Anthropic-shaped) parameter. If your policy sets a lower ceiling for a specific model or workflow, Ciralgo lowers the value in the forwarded request. This is enforcement by clamp, not refusal.

Per-workflow rolling caps. Each workflow (identified by the attribution triple) can have a per-hour or per-day spending cap in euro. Every completed request adds its cost_eur to a rolling counter. When the counter passes the cap, Ciralgo starts refusing new requests to that workflow with PROVIDER_BLOCKED and reason budget. The refusal message is explicit so applications can display a useful error to the user.

Per-organisation ceiling. An absolute monthly ceiling exists at the top of every organisation's contract. It exists so an accidental fan-out (a broken agent that calls the LLM ten thousand times) cannot generate a bill that surprises finance. This ceiling is a safety net, not a budget tool. If a workflow hits it, someone gets paged.

None of the three caps override jurisdiction. If EU-only would refuse the request, Ciralgo does not fall back to a cheaper US model just because the budget is fine.

Private-intent enforcement

Some requests are more sensitive than others. A customer support draft can go to a general-purpose model. A prompt that includes salary data, medical notes, or interview feedback should not.

Ciralgo supports two intent-signalling mechanisms, and the policy engine reacts to both.

The first is inline detection. Ciralgo runs a set of pattern detectors over every prompt before it leaves. The detectors are opt-in per organisation and can include EU-specific patterns like BSN (Dutch citizen ID), IBAN, and named-entity classes for common regulated categories. If a detector matches, the request is tagged private-intent. What happens next depends on your policy: redaction and continued routing, refusal, or forced routing to a specific private-preferred provider.

The second is explicit intent headers. Applications can send X-Ciralgo-Intent: private on a request to declare, before the fact, that the payload is sensitive. This is preferred over inline detection when the calling application already knows the classification (for example, an HR tool that always handles regulated data). Explicit intent triggers policy without a detector pass, which is faster and does not depend on pattern quality.

Both mechanisms produce a REDACTION_APPLIED or POLICY_APPLIED GovernanceEvent, and the resulting provider_region on the UsageRecord tells the audit trail which region the private request actually landed in.

Allowlists and blocklists

Every organisation ships with an implicit allowlist: only the providers registered on the account are reachable. Adding a new provider requires a change ticket on the account, and the change itself produces a governance record.

Within the allowlist, the policy can further restrict. A common shape is:

• Global allowlist: OpenAI, Anthropic, Mistral, OVHcloud, Google Vertex.
• sensitive-workflows team: Anthropic and OVHcloud only. Explicitly excludes OpenAI and Google.
• marketing project inside the growth team: OpenAI only. Everything else refused.

The policy engine walks from the widest to the narrowest scope: workflow, project, team, organisation. The first match wins. If nothing matches, the global allowlist applies.

Blocklists are the inverse and are usually short-lived. If a provider has a security incident, the account owner can push a blocklist entry naming the provider and a reason, which then applies to every workflow immediately. Blocklists are visible in the audit log and expire on a fixed timestamp unless renewed.

Fallback semantics

Ciralgo disables fallback by default. The design assumption is that a workflow that names openai/gpt-4o wants OpenAI GPT-4o and nothing else, and would rather fail than silently talk to a different model.

An organisation can enable fallback per workflow or per project. You declare the fallback explicitly: "if OpenAI is unavailable for this workflow, try Anthropic Claude Sonnet 4.6, then Mistral Large, then refuse". Every step in the chain writes a POLICY_APPLIED event so an auditor can see which fallbacks fired and why.

Two guardrails apply even when you enable fallback.

First, fallback never crosses jurisdiction. If the primary is EU-only, the fallback chain must also be EU-only. Specifically, the policy engine rejects any US fallback for an EU-only primary at policy-load time.

Second, fallback never crosses sensitivity level. A workflow that requires X-Ciralgo-Intent: private cannot fall back to a provider you have not marked private-safe.

These two guardrails exist to prevent the exact silent-degradation failure mode that makes AI Act audits fail.

The ProviderRoutingDecision record

Every completed request produces a ProviderRoutingDecision record alongside its UsageRecord. The two are correlated by request_id. The routing record has three parts.

Inputs. Snapshot of the four inputs at request time: the policy version in force, the requested model, the intent tag if present, the workflow attribution triple, the provider availability snapshot.

Rules fired. Ordered list of every rule that matched, with the outcome of each. A typical simple request has one entry: policy_v42:jurisdiction-eu-only matched, permitted. A complex request may have six or seven, ending with the winning rule.

Winner. The provider, region, and model that actually served the request. If the request was refused, the winner is refused with a reason code.

The three parts together let an auditor answer the question "why this provider for this request" without needing access to the current policy configuration. The policy version is captured by hash, and the hash is stable across time. Even if the policy changes tomorrow, the record from today still points at the exact rules that fired then.

Common patterns customers configure

The following patterns show up regularly. They are not opinions from Ciralgo; they are how existing customers actually run their platforms.

Development-and-production split. Development workloads may reach US regions and cheaper providers because the data is synthetic. In contrast, production workloads stay EU-only. Which Ciralgo key you use in each environment enforces the split. Records then show clearly which environment produced each request.

High-sensitivity workflow group. You group HR, legal, medical, and executive workflows under a single team called sensitive. That team's LLM provider routing policy stays EU-only, Anthropic-first for reasoning, OVHcloud-first for retrieval augmentation, no fallback, no cost cap. Cost is not the concern; leakage is.

Model swapping for research. A specific research project can request anthropic/claude-opus-4-7 and openai/gpt-4.5-preview on the same day. Its policy allows both, subject to jurisdiction. The audit trail then shows which the workflow actually used, which lets researchers compare cost and latency by policy-neutral measurement.

GPAI carve-out. Customers whose downstream customers require GPAI-only handling can add a policy that refuses fine-tuned models and only permits general-purpose ones. Meanwhile, Ciralgo distinguishes the two on the provider surface and applies the carve-out at request time.

Frequently asked questions about LLM provider routing policy

Can we run our own routing logic and use Ciralgo just as a passthrough?

Yes. Set your LLM provider routing policy to allow every reachable provider with no additional rules, and your application picks a model via the openai/*, anthropic/*, or provider-prefixed model string. Ciralgo still produces the UsageRecord and the ProviderRoutingDecision, because the audit obligation does not disappear when the routing is trivial.

How fast does a policy change take effect?

Ciralgo applies new policy versions within seconds of saving. Meanwhile, requests already in flight complete under the version that was current when they arrived. As a result, the routing record captures the exact policy hash the request used, so a policy change never rewrites history.

What if OpenAI adds an EU Data Zone for a model we did not previously have EU access to?

Ciralgo detects the availability change and updates the provider surface. As a result, your existing EU-only policy for that model then routes into the new region automatically. You do not need to reconfigure anything. For OpenAI in particular, Data Zones for enterprise customers is the relevant contractual layer.

Can policies reference workload types other than the attribution triple?

Yes. Specifically, policies can also match on the model string, the intent header, the size of the request in tokens, and on custom labels the calling application sends. In practice, custom labels are useful when the attribution triple is not granular enough (for example, distinguishing "internal-user" from "external-user" workflows inside a single team).

What is the difference between a policy refusal and a provider refusal?

A policy refusal happens inside Ciralgo before the request leaves. It produces PROVIDER_BLOCKED with a reason. By contrast, a provider refusal happens after Ciralgo forwards the request, when the underlying model refuses to answer (content policy, invalid input, and so on). Consequently, it comes back through Ciralgo as the provider's own error, with terminated: refused_by_provider on the UsageRecord.

Is the LLM provider routing policy human-readable?

Yes. Policies are declarative JSON with a schema. Furthermore, every policy version is diffable against the previous version, so change review is straightforward. As a result, compliance officers can read a policy without any translation.

Do we need to change our policy when a new provider or model is added?

Only if your policy uses allowlist scoping. Specifically, deny-by-default policies pick up new providers automatically, while explicit-allowlist policies do not. In practice, most regulated deployments use explicit allowlists.

Related reading

• AI audit logging under the EU AI Act
• Connecting to the Ciralgo proxy
• What is an EU-hosted LLM proxy?
• Ciralgo security whitepaper

Last refreshed: 9 July 2026. This page is reviewed quarterly.