Provider Routing

Many models are sold by more than one provider. gpt-5.6-sol is served by four providers, kimi-k2.6 by seven, gpt-oss-120b by fifteen. They run the same weights at different prices, speeds and reliability. Name a model without a provider prefix and Eden AI picks which provider serves it.

{
  "model": "openai/gpt-5.6-sol"
}

Pins the provider. Eden AI calls OpenAI, and only OpenAI.

{
  "model": "gpt-5.6-sol"
}

Leaves the choice to Eden AI, which selects among every provider serving that model. A model sold by a single provider routes to that provider whether or not you name it. Provider routing only has a decision to make where two or more providers offer the same model.

Finding the routable models

Ask any model listing for the grouped view and you get one entry per routable name, with the providers behind it nested underneath:

cURL

curl "https://api.edenai.run/v3/models?view=models"

{
  "id": "gpt-5.6-sol",
  "object": "model",
  "owned_by": "openai",
  "mode": "chat",
  "endpoint_count": 8,
  "endpoints": [
    { "id": "openai/gpt-5.6-sol", "owned_by": "openai", "...": "pricing, capabilities, context_length, regions" },
    { "id": "azure/gpt-5.6-sol",  "owned_by": "azure",  "...": "pricing, capabilities, context_length, regions" },
    "...6 more: Amazon, Databricks and region-pinned endpoints such as azure/gpt-5.6-sol@eu"
  ]
}

The name level makes no claim about price, context window or capability, because providers of one model genuinely differ on all three. Each entry in endpoints carries its own, so this one call answers both questions: which names route, and what each provider charges and supports. See Listing Models for the full field reference.

How a provider is chosen

By default Eden AI optimises for cost: the cheapest provider for the shape of your request, accounting for how much of it is prompt versus completion. Selection is weighted rather than absolute: cheaper providers receive proportionally more traffic instead of every request piling onto a single one. That keeps you off one provider’s rate limits and spreads exposure when a provider degrades. Providers that are failing are ranked below healthy ones and are never chosen first. They stay in the chain as a last resort, because a struggling provider is still better than no answer.

Choosing an objective

Set routing.sort to optimise for something other than price.

Value	Optimises for
`cost`	Cheapest for this request. Default
`speed`	Highest observed throughput
`latency`	Lowest observed time-to-first-token
`exact`	Best instruction-following, for tool calls and structured output

{
  "model": "gpt-5.6-sol",
  "messages": [{"role": "user", "content": "Summarise this contract"}],
  "routing": {"sort": "cost"}
}

The same objective can be written as a suffix on the model name, which is useful when a client only lets you configure a model string:

{
  "model": "gpt-5.6-sol:latency"
}

speed, latency and exact rank on observed performance. When Eden AI has too little data for a model, routing falls back to ranking on price rather than guessing, and your request still succeeds.

Naming an objective also turns off traffic spreading: cost means the cheapest provider every time, not a weighted draw. sort never changes which model answers, only who serves it.

Trying providers in a set order

Set routing.order to name the providers to try first, in the order you want them tried.

{
  "model": "gpt-5.6-sol",
  "messages": [{"role": "user", "content": "hi"}],
  "routing": {"order": ["azure", "openai"]}
}

The providers you list are tried first, in your order. The model’s other providers follow as backups, cheapest first, or ranked by your sort objective if you set one. Traffic is not spread by price: once your list matches a provider, the chain is a fixed ranking, not a weighted draw.

A provider marked down moves last. If health checks mark a listed provider as down, it moves to the end of the chain, whatever its position in your list.
order is a preference, not a restriction. If none of the providers you list sells the model, or all of them are marked down, the request starts with another provider of the model. With allow_fallbacks: false, that provider is the only one tried. To make sure only certain providers are ever used, list them in allowed_providers too.
Long lists may not be tried in full. A request makes at most 8 attempts, and the entries in your fallbacks keep their slots first. See Fallback.
An entry that does not sell the model is skipped. Naming any order still turns off sticky routing, as an explicit sort does, even when none of its entries sell the model.

Restricting which providers may be used

routing.allowed_providers narrows the pool to providers you trust or have agreements with. Everything else is excluded, including from failover.

{
  "model": "gpt-5.6-sol",
  "messages": [{"role": "user", "content": "hi"}],
  "routing": {"allowed_providers": ["openai"]}
}

If no provider in the list serves the requested model, the request fails with a clear error rather than silently routing elsewhere.

Excluding specific providers

Set routing.ignored_providers to keep a request away from specific providers.

{
  "model": "kimi-k2.6",
  "messages": [{"role": "user", "content": "hi"}],
  "routing": {"ignored_providers": ["deepinfra"]}
}

This is the mirror of allowed_providers: every provider of kimi-k2.6 except DeepInfra can serve the request, including on failover. An entry that does not sell the model has no effect. If the list excludes every provider of the model, the request fails with a 400 naming the providers that sell it.

Setting a maximum price

Set routing.max_price to route only to providers whose listed per-token rates are at or below your ceiling.

{
  "model": "gpt-5.6-sol",
  "messages": [{"role": "user", "content": "hi"}],
  "routing": {
    "max_price": {
      "input": 4,
      "output": 20
    }
  }
}

At today’s list prices, this keeps OpenAI, Azure and Databricks (

4 input,

20 output) and skips Amazon (

4.40 input,

22 output). Each provider’s rates are in the pricing of its entry in /v3/models?view=models. Prices are in USD per million tokens. Set input, output, or both. Providers that charge by another unit (such as per image or per second) are not filtered. If no provider of the model is within the ceiling, the request fails with a 400 naming the cheapest provider and its rates, so you know what ceiling would work.

How these settings apply

order, allowed_providers, ignored_providers and max_price follow the same rules:

Use the provider name from the catalog. It is the part before the first / in an endpoint id from /v3/models?view=models, such as amazon (not bedrock), vertex, fireworks_ai or together_ai. Names are case-insensitive. A name that matches no provider is not an error, it just matches nothing, so a misspelled ignored_providers entry excludes nothing.
They apply to every model name without a provider, in model and in fallbacks. If any of them is left with no provider, the whole request fails with a 400, even when the other models still have one.
A concrete provider/model is never filtered or reordered. It goes exactly where you sent it, in model or in fallbacks. To keep a request away from a provider, don’t name that provider in either.
Invalid values return a 422: an empty provider name or one longer than 64 characters, more than 16 entries in order, the same provider in both allowed_providers and ignored_providers, or a max_price with neither input nor output, a negative price, or any other key (a typo such as ouput is rejected rather than ignored).

Requiring zero data retention

Set routing.zero_data_retention to true to serve the request only from providers that keep no prompts or outputs. Eden AI also saves neither the request nor its response.

{
  "model": "deepseek-v4-flash",
  "messages": [{"role": "user", "content": "hi"}],
  "routing": {"zero_data_retention": true}
}

It applies to every model in the request, a concrete provider/model and your fallbacks included. A model name keeps only its providers that qualify; a concrete provider/model or a fallback that doesn’t qualify fails the request with no_compliant_provider, naming each one and why. Which models pass, how your own keys are handled, what else it turns off, and how an organization requires it for every call: see Compliance Policies.

Disabling provider failover

By default, if the chosen provider fails, Eden AI tries another provider of the same model. Set routing.allow_fallbacks to false to stop that. The request is attempted once and the error is returned to you.

{
  "model": "gpt-5.6-sol",
  "messages": [{"role": "user", "content": "hi"}],
  "routing": {"allow_fallbacks": false}
}

allow_fallbacks governs other providers of the model you requested. Models you list yourself in fallbacks are your own choice and are always kept. See Fallback.

Setting allow_fallbacks to null is equivalent to omitting it, leaving provider failover enabled.

Keeping a conversation on one provider

A prompt cache lives at one provider endpoint. If routing picks a different provider on turn two of a conversation, that cache is not there, so you pay to build it again instead of reading it at a discount. Sticky routing keeps a conversation on the provider that already holds its cache. It is on by default, and only ever active for models whose providers discount cache reads. Where there is no discount there is nothing to gain, so routing keeps spreading by price. You don’t have to do anything: with no identifier, Eden AI recognises a conversation from its opening messages, which don’t change as it grows. Sending an identifier makes it reliable. Use any stable string: a thread id, a ticket number, an agent run id.

{
  "model": "gpt-5.6-sol",
  "session_id": "conv-7f3a91",
  "messages": [{"role": "user", "content": "Summarise this contract"}]
}

For clients that cannot add body fields, such as a coding agent that only lets you set headers, send the same value as a header. Eden AI checks x-session-id, x-claude-code-session-id, x-session-affinity, and session-id. The body field wins if headers are present.

cURL

curl -X POST https://api.edenai.run/v3/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "x-session-id: conv-7f3a91" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-5.6-sol", "messages": [{"role": "user", "content": "hi"}]}'

session_id and the session headers are accepted on all three chat dialects (/v3/chat/completions, /v3/responses and /v3/v1/messages) and are capped at 256 characters. They are never forwarded to the provider.

How long provider affinity lasts

After Eden AI observes a cache write or read, it keeps the conversation on that provider for 10 minutes by default. Each later cache write or read refreshes the window. When you declare a prompt-cache lifetime, sticky routing follows it so the provider affinity matches the cache window:

Cache setting	Provider affinity
No duration, or `prompt_cache_retention: "in_memory"`	10 minutes
`cache_control.ttl: "5m"`	5 minutes
`prompt_cache_options.ttl: "30m"`	30 minutes
`cache_control.ttl: "1h"`	Up to 1 hour for Anthropic and Bedrock
`prompt_cache_retention: "24h"`	Up to 24 hours for OpenAI and Azure

If a request declares more than one duration, Eden AI uses the longest one, capped at the maximum retention of the provider that served the cache write or read. Providers without a known extended-retention policy use a conservative one-hour ceiling.

Choosing a session id

The identifier has to be the same on every turn of one conversation. That is the whole contract, and both ways of getting it wrong look like using the feature correctly:

	Identifier	What happens
✅	A thread id, ticket number, or agent run id	Stable for the conversation’s life. One cache write, then reads
❌	A fresh UUID per request	A new conversation every turn, so every turn re-routes. You opted in and got nothing
❌	The end user’s account id	All of that user’s conversations collide on one provider. Different prompts, so no cache benefit, and their traffic stops spreading by price

Mint it once when the conversation starts, store it alongside the conversation, and send it on every turn.

Turning it off

{
  "model": "gpt-5.6-sol",
  "messages": [{"role": "user", "content": "hi"}],
  "routing": {"sticky": false}
}

Naming an explicit routing.sort or routing.order also takes priority over cache affinity. And a concrete provider/model was never routed in the first place.

Confirm it is working with Request Metadata: the provider in summary should be the same on every turn. Your usage block reports the cached tokens. Expect none on the first turn, since there was nothing to read yet.

Pinning a region

Append @region to route to a provider endpoint in a specific region. This composes with everything above, and each entry in a chain carries its own region, so the primary and the fallback below are two separate attempts in two separate regions.

{
  "model": "<model>@eu",
  "fallbacks": ["<model>@us"]
}

Which regions a model offers varies by model. Requesting one it is not served from returns an error rather than quietly serving it elsewhere. See Servers Location.

Seeing which provider served your request

Routing is invisible by default: the response looks the same whichever provider answered. Send x-edenai-metadata: enabled and Eden AI attaches what it decided, including every provider it tried and the status each returned.

"edenai_metadata": {
  "strategy": "routed",
  "summary": "available=2, served=openai/gpt-5.6-sol",
  "attempt": 1,
  "attempts": [
    {"provider": "openai", "model": "openai/gpt-5.6-sol", "status": 200, "region": "global"}
  ]
}

See Request Metadata for every field, the strategy values, and how to read the block off a stream.

Turning routing off

Name a provider/model and routing never runs. The request goes exactly where you sent it. This is the behaviour of every request that names a provider, and nothing about provider routing changes it.

Next Steps

Fallback

Name your own backup models

LLM Models

Browse available models and their pricing

Servers Location

Where each region runs

Request Metadata

See which provider actually served the request

V3 Documentation

Quick Start

Overview

LLMs

Expert Models

General

Organization

Data Governance

Integrations

Finding the routable models

How a provider is chosen

Choosing an objective

Trying providers in a set order

Restricting which providers may be used

Excluding specific providers

Setting a maximum price

How these settings apply

Requiring zero data retention

Disabling provider failover

Keeping a conversation on one provider

How long provider affinity lasts

Choosing a session id

Turning it off

Pinning a region

Seeing which provider served your request

Turning routing off

Next Steps

Fallback

LLM Models

Servers Location

Request Metadata

​Finding the routable models

​How a provider is chosen

​Choosing an objective

​Trying providers in a set order

​Restricting which providers may be used

​Excluding specific providers

​Setting a maximum price

​How these settings apply

​Requiring zero data retention

​Disabling provider failover

​Keeping a conversation on one provider

​How long provider affinity lasts

​Choosing a session id

​Turning it off

​Pinning a region

​Seeing which provider served your request

​Turning routing off

​Next Steps

Fallback

LLM Models

Servers Location

Request Metadata

Finding the routable models

How a provider is chosen

Choosing an objective

Trying providers in a set order

Restricting which providers may be used

Excluding specific providers

Setting a maximum price

How these settings apply

Requiring zero data retention

Disabling provider failover

Keeping a conversation on one provider

How long provider affinity lasts

Choosing a session id

Turning it off

Pinning a region

Seeing which provider served your request

Turning routing off

Next Steps