How to route between OpenAI and Anthropic cheaply, with audit logs intact

Learning how to route between OpenAI and Anthropic cheaply, without giving up the audit log your CFO and your DPO both need, is one of the most common practical problems that arrives the moment a team adds a second AI provider. On paper it looks like a pure cost optimisation: pick whichever model is cheapest for each workflow. In practice, the "cheapest" answer breaks down as soon as you factor in the fallback path, the context-window pricing asymmetry, and the specific log fields an EU AI Act audit will ask for. This guide describes what "cheapest" actually means when you route between OpenAI and Anthropic, three routing strategies that survive contact with reality, and the small set of audit-log fields you must capture regardless of which strategy you pick.

The audience is the engineer or founder who is already paying two AI providers and wants to route intelligently between them. Team size does not gate this problem: a three-developer team splitting workloads across OpenAI and Anthropic hits routing complexity on day one, often more chaotically than a larger team where a platform group has already imposed defaults.

Route between OpenAI and Anthropic: what "cheapest" actually means

Cheapest is three costs, not one. When you calculate the true cost per workflow of routing between OpenAI and Anthropic, only the first is on the provider's pricing page:

1. Base per-token cost. Input tokens plus output tokens times the per-1K price of the model actually called. Easy to read off OpenAI's pricing page or Anthropic's pricing page but only one part of the answer.
2. Fallback cost. When your primary provider rate-limits or times out, the retry lands on your fallback. If you route Anthropic-first and fall back to OpenAI on error, your average cost per request is a weighted mix of both, not the sticker price of the cheaper one.
3. Complexity cost. Engineering time to maintain the routing logic, plus the additional log fields the multi-provider setup forces you to capture. Usually pays back but often gets forgotten in the initial comparison.

A team that only optimises for the first cost typically overpays on the second and underinvests in the third, leaving the finance team with a spend they cannot attribute and the compliance team with an audit trail that stops at the routing layer.

Model tiers on both providers, in ratios that survive price changes

Provider prices move often enough that specific numbers age within weeks, but the relative cost structure is stable. As a rough guide:

• Low tier (classification, extraction, simple chat): Claude Haiku on Anthropic, GPT-4o-mini or GPT-4.1-mini on OpenAI. Typically 5 to 10 times cheaper per 1K tokens than the next tier up.
• Mid tier (general assistants, summarisation, most agent workflows): Claude Sonnet on Anthropic, GPT-4o on OpenAI. Roughly 2 to 5 times cheaper than the top tier and where most SME workloads run.
• Top tier (deep reasoning, complex agents): Claude Opus on Anthropic, o1 or o3 on OpenAI. Reserved for workflows where cheaper models measurably underperform.

Output tokens cost more than input tokens on both providers, typically 3 to 5 times more. That asymmetry matters: a workflow that reads a 10K-token document and returns 200 tokens is dominated by the input cost; a workflow that generates 5K tokens from a short prompt is dominated by the output cost. Route based on which side of that asymmetry your workflow sits on, not on the provider's headline "per 1M" number.

Three strategies to route between OpenAI and Anthropic

Each of the three practical strategies below trades routing sophistication for engineering cost. Pick based on how many distinct workflow types you have.

Strategy 1: cheapest-tier per workflow

Map each workflow to whichever provider is cheapest at the tier that workflow requires. Classification tasks go to Haiku or GPT-4o-mini, general chat goes to Sonnet or GPT-4o, deep reasoning goes to Opus or o1. Providers do not compete evenly at every tier, so the "cheapest" often flips between tiers.

• Best for: five or fewer distinct workflow types where the model requirement is stable
• Log implication: capture provider and model_actually_used per request; both fields are essential for later cost analysis

Strategy 2: primary-fallback with a cost cap

Set a primary provider for each workflow tier and a fallback for rate limits, errors, or region outages. When the primary responds successfully, cost matches the sticker price. When it fails, you pay the fallback price plus one wasted request. Cap the fallback ratio so a bad day at one provider does not silently double your monthly bill.

• Best for: workloads with real availability sensitivity (customer-facing chat, CI on the critical path)
• Log implication: capture the fallback chain per request. If your log only records the provider that succeeded, you lose visibility into how much you paid for failed primaries. Include a fallback_reason field so cost anomalies trace back to a specific provider incident.

Strategy 3: latency-aware cheapest route

Choose the cheapest model that meets your latency budget. Some cheap models are slower per token; in high-volume workloads the latency budget shrinks and the cheap option stops being viable. This strategy requires per-request latency measurements and a scheduler willing to skip Haiku or 4o-mini when the queue is deep.

• Best for: high-volume real-time workflows (search-suggestion, autocomplete, live translation)
• Log implication: capture latency_p50 and latency_p95 per provider per model per hour, plus the routing decision reason ("cheapest under 800ms budget"). Costly to instrument but usually the difference between a good and bad multi-provider setup at scale.

The audit-log fields that matter regardless of strategy

Cheapest routing is worthless if the audit trail is thin, because the same finance team that wants a lower invoice will want to defend it in a board meeting. Six fields have to be present on every request, across every provider you route to:

1. Provider name (openai, anthropic, mistral) so cross-provider reports work at all
2. Model actually used (not requested, not aliased) because failovers change which model handled the request
3. Region and jurisdiction where the request executed, per EU AI Act obligations for European SMEs
4. Team and user identity so cost attribution and access review both work off one source (see how to attribute AI costs to teams)
5. Token counts (input, output, and when available, cached input) since asymmetry between input and output cost dominates the actual bill
6. Fallback chain listing which providers were tried and why the failed ones failed, so overspend never looks like unexplained variance

Missing any of these turns the multi-provider setup into a black box that finance cannot audit and compliance cannot defend.

Small multi-provider teams face the same problem as large ones

Team size does not gate the routing problem. A three-developer team paying both OpenAI and Anthropic hits the same three-cost calculus as a fifty-developer team, and the small team usually has less discipline about tagging requests consistently because there is no platform team to enforce it. The most common failure mode at small scale is one developer picking a cheaper model without telling anyone, one workflow silently falling back to the more expensive provider for two weeks, and the CFO discovering a €1800 anomaly at month-end that no dashboard can explain.

A proxy layer solves this at any size by centralising the routing decision, normalising the audit-log fields across providers, and producing one report instead of three. Ciralgo's EU-hosted LLM proxy is designed for exactly this pattern; the provider routing governance page describes the decision rules in more detail.

Common mistakes when routing between OpenAI and Anthropic

Four failure modes recur.

Optimising for headline per-1K token price only. The sticker number is a starting point, not the answer. Context-window pricing, cached-input pricing on both providers, and the input/output asymmetry can flip the "cheapest" answer for your specific workflow.

Losing the fallback chain in the audit log. If your logs only record the successful provider, you lose visibility into how much you paid for failed primaries. This is the single most common gap when teams first set up multi-provider routing.

Routing sensitive data to whichever is cheap without a jurisdiction check. For European SMEs, a cheap US-hosted call for a workflow with an EU-residency requirement is not a saving. It is a compliance breach that carries a tag. Every routing decision must run a jurisdiction check before the cost comparison.

Assuming vendor lock-in makes routing impossible. Both OpenAI and Anthropic offer chat-completion APIs with enough overlap that switching for a specific workflow is usually a few lines of code, especially when a proxy handles the request-shape translation. The lock-in argument is weaker than it was two years ago.

Related reading

• What is an EU-hosted LLM proxy: the infrastructure layer that centralises the routing decision.
• LLM provider routing policy: how the decision is governed at Ciralgo, aimed at the person signing the DPA.
• AI audit logging: a deeper look at the log fields the EU AI Act expects.
• How to attribute AI costs to teams: the attribution counterpart, since routing decisions produce the data that attribution consumes.

Frequently asked questions

Is Anthropic always cheaper than OpenAI for the same workload?

No. Both providers compete tier by tier, and the cheapest option flips depending on whether you need low-tier extraction, mid-tier chat, or top-tier reasoning. Anthropic tends to lead on cached-input pricing for repeated context, OpenAI tends to lead on the low tier for short prompts. Route by workflow, not by provider preference.

Do I need a proxy layer to route between OpenAI and Anthropic?

Not strictly. You can hard-code routing in each application that calls an LLM. A proxy becomes worth the setup as soon as more than one application needs to make the same decision, or as soon as you want the audit log fields normalised across both providers in one place. Small multi-provider teams often reach this point immediately, since duplicating routing logic across two or three services is more effort than adopting a proxy.

How do I know if a request was routed via fallback?

Only if your log captures the fallback chain. If it captures only the provider that returned a successful response, you cannot distinguish "routed to OpenAI first, succeeded" from "routed to Anthropic first, failed, OpenAI succeeded on retry". These have different costs. Add a fallback_reason field to every log record.

What audit log fields matter most for the EU AI Act?

Provider, model, region/jurisdiction, team identity, token counts, and the fallback chain. Missing any of these leaves the audit trail thin enough that a conformity assessment reviewer will ask for it in a follow-up round. Better to capture from day one than retrofit later.

Can I A/B test between providers on the same workflow?

Yes, and it is often the right way to decide which provider actually performs best for your workflow. Route a fraction of production traffic to the alternative provider, capture the same audit-log fields for both, and compare accuracy plus cost after a week. This only works cleanly if the audit log fields are consistent across providers, which is what a proxy layer normalises.