Connecting to the Ciralgo proxy

EU LLM proxy integration is a one-line change in most modern SDKs. Ciralgo speaks the OpenAI and Anthropic REST protocols on the wire, so switching an existing application from a direct provider to a Ciralgo-routed setup means updating a single base-URL string. Everything else, including the SDK behaviour, the response shape, and the streaming semantics, stays exactly the same.

That is the design goal of an EU LLM proxy integration: applications should not need to know they are talking to a proxy. In practice, the proxy quietly routes to an EU-hosted endpoint, applies your organisation's data and access rules, and produces the audit records described in AI audit logging under the EU AI Act. Meanwhile, your code keeps calling the SDK you already trust.

This page covers the 60-second EU LLM proxy integration quickstart, followed by complete examples in Python, Node, and Go, then the attribution headers that make records useful for finance and compliance, and finally the common mistakes teams hit in the first day.

EU LLM proxy integration in 60 seconds

Three steps. That is all.

1. Get a Ciralgo API key from your organisation's admin. It looks like a long opaque string, same shape as an OpenAI key.
2. In your SDK, change the base URL to point at Ciralgo instead of the direct provider.
3. Optional but recommended, add three request headers that identify the team, project, and workflow. Those headers drive per-team cost attribution and audit records.

The endpoints are:

• OpenAI-compatible traffic: https://api.ciralgo.com/v1
• Anthropic-compatible traffic: https://api.ciralgo.com/anthropic/v1
• Direct model access for OpenAI-shaped requests to non-OpenAI models: https://api.ciralgo.com/v1 with the model name of the target provider (see the routing section below).

The Ciralgo key you use is the same across all three surfaces. Ciralgo internally maps the key to the workloads it is allowed to reach.

Python: OpenAI SDK

The official openai Python SDK, version 1.0 or newer, takes a base_url on the client. That is the only change from a direct-OpenAI setup for your EU LLM proxy integration.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_CIRALGO_KEY",
    base_url="https://api.ciralgo.com/v1",
    default_headers={
        "X-Ciralgo-Team": "growth",
        "X-Ciralgo-Project": "onboarding-bot",
        "X-Ciralgo-Workflow": "welcome-message",
    },
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Introduce Ciralgo in one sentence."},
    ],
)

print(response.choices[0].message.content)

Everything else the OpenAI SDK exposes, including streaming, function calling, structured outputs, and image inputs, works unchanged because Ciralgo forwards the request shape verbatim.

Python: Anthropic SDK

The official anthropic Python SDK, version 0.30 or newer, takes a base_url on the client.

from anthropic import Anthropic

client = Anthropic(
    api_key="YOUR_CIRALGO_KEY",
    base_url="https://api.ciralgo.com/anthropic/v1",
    default_headers={
        "X-Ciralgo-Team": "risk",
        "X-Ciralgo-Project": "policy-summariser",
        "X-Ciralgo-Workflow": "manager-review",
    },
)

message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Summarise this policy in three bullets."},
    ],
)

print(message.content[0].text)

Model identifiers are pass-through. So if Anthropic ships a new Claude version, you can call it through Ciralgo the day it is available. By default, the proxy does not enforce a model allowlist, although your organisation's routing policy may.

Python: LangChain

LangChain wraps OpenAI and Anthropic. Both wrappers accept an override for the base URL, which is all an EU LLM proxy integration needs.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o",
    api_key="YOUR_CIRALGO_KEY",
    base_url="https://api.ciralgo.com/v1",
    default_headers={
        "X-Ciralgo-Team": "product",
        "X-Ciralgo-Project": "docs-search",
        "X-Ciralgo-Workflow": "retrieval",
    },
)

print(llm.invoke("What does Article 12 of the EU AI Act require?").content)

For Anthropic via LangChain, use langchain_anthropic.ChatAnthropic with the same base_url override pattern. LangChain retries, streaming, and callbacks continue to work unchanged.

Python: LlamaIndex

LlamaIndex uses the same OpenAI wrapper under the hood, so the pattern is identical.

from llama_index.llms.openai import OpenAI as LlamaOpenAI

llm = LlamaOpenAI(
    model="gpt-4o",
    api_key="YOUR_CIRALGO_KEY",
    api_base="https://api.ciralgo.com/v1",
    default_headers={
        "X-Ciralgo-Team": "sales",
        "X-Ciralgo-Project": "rfp-assistant",
        "X-Ciralgo-Workflow": "draft-response",
    },
)

response = llm.complete("Draft a one-paragraph reply to this RFP question.")
print(response.text)

Note that the parameter is api_base, not base_url, in LlamaIndex's wrapper. As a result, this is a common paste-error source.

Node.js: OpenAI SDK

The Node openai SDK, version 4.0 or newer, takes a baseURL on construction (camelCase, not snake_case).

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.CIRALGO_KEY,
    baseURL: "https://api.ciralgo.com/v1",
    defaultHeaders: {
        "X-Ciralgo-Team": "engineering",
        "X-Ciralgo-Project": "codegen-assistant",
        "X-Ciralgo-Workflow": "review-suggestions",
    },
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [
        { role: "user", content: "Explain semaphores to a Rust developer." },
    ],
});

console.log(response.choices[0].message.content);

Streaming with stream: true and the async iterator pattern works unchanged.

Node.js: Anthropic SDK

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
    apiKey: process.env.CIRALGO_KEY,
    baseURL: "https://api.ciralgo.com/anthropic/v1",
    defaultHeaders: {
        "X-Ciralgo-Team": "support",
        "X-Ciralgo-Project": "ticket-triage",
        "X-Ciralgo-Workflow": "category-suggest",
    },
});

const message = await client.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 1024,
    messages: [
        { role: "user", content: "Classify this ticket into one of five categories." },
    ],
});

console.log(message.content[0].text);

Go: raw net/http

Unlike Python and Node, Go does not have an officially maintained OpenAI SDK. Most teams therefore use github.com/sashabaranov/go-openai, which supports a custom base URL, or raw net/http. Either way, both approaches work as an EU LLM proxy integration without change.

package main

import (
    "context"
    "fmt"

    openai "github.com/sashabaranov/go-openai"
)

func main() {
    config := openai.DefaultConfig("YOUR_CIRALGO_KEY")
    config.BaseURL = "https://api.ciralgo.com/v1"

    client := openai.NewClientWithConfig(config)

    resp, err := client.CreateChatCompletion(
        context.Background(),
        openai.ChatCompletionRequest{
            Model: openai.GPT4o,
            Messages: []openai.ChatCompletionMessage{
                {
                    Role:    openai.ChatMessageRoleUser,
                    Content: "Summarise this in one line.",
                },
            },
        },
    )
    if err != nil {
        panic(err)
    }
    fmt.Println(resp.Choices[0].Message.Content)
}

The go-openai library does not have a first-class default_headers option; add the attribution headers per request via the RequestHeaders field on the request struct, or by wrapping the HTTP client's RoundTripper.

Attribution headers

The three optional headers Ciralgo reads on every request are:

• X-Ciralgo-Team — the internal team or business unit that owns this workload. Free-form string. Used for cost attribution.
• X-Ciralgo-Project — the product or project the workload belongs to. Free-form string. Used for cost attribution and Article 26 oversight records.
• X-Ciralgo-Workflow — the specific workflow within the project. Free-form string. Used for accuracy tracking and refusal rate analysis.

Technically, none of the three is required for the request to succeed. However, all three are recorded in the UsageRecord if present. As a result, finance teams typically ask for these to be populated on day one, because without them cost attribution collapses into a single monthly bill.

In practice, the cleanest EU LLM proxy integration pattern is to inject these headers via a helper. For example, in Python:

from openai import OpenAI

def ciralgo_client(team: str, project: str, workflow: str) -> OpenAI:
    return OpenAI(
        api_key=os.environ["CIRALGO_KEY"],
        base_url="https://api.ciralgo.com/v1",
        default_headers={
            "X-Ciralgo-Team": team,
            "X-Ciralgo-Project": project,
            "X-Ciralgo-Workflow": workflow,
        },
    )

Every workflow constructs its own client at startup with the correct attribution triple. Consequently, requests never carry stale or wrong attribution because clients are single-purpose.

Choosing which provider a request lands on

By default, Ciralgo routes based on the model field in your request plus the routing policy your organisation has configured. For example, gpt-4o traffic forwards to OpenAI in the region your policy allows. When the request names claude-sonnet-4-6, it lands on Anthropic via AWS Bedrock in Frankfurt. Meanwhile, mistral-large traffic goes to Mistral in France.

You can pin a specific provider for a request by using a fully-qualified model identifier:

• openai/gpt-4o — always OpenAI, subject to policy.
• anthropic/claude-sonnet-4-6 — always Anthropic.
• mistral/mistral-large — always Mistral.
• ovh/mixtral-8x7b-instruct — always OVHcloud.

You can also request a specific provider region using a header:

X-Ciralgo-Provider-Region: eu-west-1

Full policy semantics live in Provider routing governance.

Common mistakes on day one

Mistake	Symptom	Fix
Using api_base when the SDK expects base_url	TypeError: unexpected keyword argument on client construction	LlamaIndex uses api_base, most other SDKs use base_url or baseURL. Check the exact spelling for the wrapper you use.
Missing the /v1 suffix on the base URL	404 on every request	The base URL must end in /v1 for OpenAI-shaped traffic or /anthropic/v1 for Anthropic-shaped.
Sending an OpenAI key by mistake	401 unauthorized	Ciralgo does not proxy provider keys, it uses its own. Set CIRALGO_KEY, not OPENAI_API_KEY.
Forgetting the attribution headers in production	Finance cannot attribute cost by team	Inject the headers at client construction, not per-request.
Assuming a model is EU-hosted when it is not	Data leaves the EU without a Ciralgo policy blocking it	Configure the routing policy per workload. See the provider routing governance page.
Streaming works locally but drops in production	Idle connection killed by an intermediate proxy	Increase your load balancer's idle timeout to at least 90 seconds for streaming endpoints.
Very long completions time out	Client-side timeout hits before the model finishes	Configure a generous per-request timeout and a client-side circuit breaker as described in Timeouts and circuit breakers.

Frequently asked questions

Do I need a separate Ciralgo key per environment?

Yes. Development, staging, and production should each have their own key. In addition, Ciralgo enforces environment-level isolation on records and rate limits based on the key. Otherwise, sharing a key across environments makes cost attribution ambiguous and mixes production traffic with test traffic in your audit log.

Is there a rate limit?

Yes, but it is generous by default and is per-organisation, not per-key. However, if you have unusual peak traffic, tell your account owner in advance. In addition, the proxy transparently forwards provider-side rate limits when they occur, so a 429 from OpenAI comes through as a 429 with the original headers preserved.

Does Ciralgo support all model features?

Yes, for the models Ciralgo routes to. Specifically, streaming, function calling, structured outputs, tool use, vision inputs, and JSON mode all pass through unchanged. Furthermore, the proxy does not transform the request or response body except to inject and remove its own headers. As a result, if a feature works when calling the provider directly, it works when calling through Ciralgo.

What happens if the primary provider is down?

Depending on your routing policy, either the request fails cleanly with the provider's own error, or Ciralgo falls back to a configured secondary provider. By default, fallback is off, because silently switching providers can violate a jurisdiction rule. However, you can enable it explicitly per workload if the workload is provider-agnostic.

Can I use a self-hosted model through Ciralgo?

Yes. Specifically, Ciralgo supports any OpenAI-compatible endpoint as a routable provider, including on-premise deployments of Llama, Mistral, or DeepSeek. After you register the endpoint with your account owner, it becomes selectable in the routing policy.

How do I test integration locally without a live Ciralgo key?

Point the SDK at https://api.ciralgo.com/v1 with a special mock key issued by your admin. In this mode, all requests return canned responses without hitting a real provider. Meanwhile, the audit records are still produced, so you can also test your logging pipeline.

What SDK versions are supported?

The proxy is protocol-compatible with any OpenAI SDK from version 1.0 onwards and any Anthropic SDK from version 0.30 onwards. However, older versions of the OpenAI SDK, before the client class refactor, will not accept a base_url cleanly and are therefore not supported.

Related reading

• What is an EU-hosted LLM proxy?
• AI audit logging under the EU AI Act
• Provider routing governance
• Timeouts and circuit breakers

Last refreshed: 9 July 2026. This page is reviewed quarterly.