Skip to main content

Module pool

Module pool 

Source
Expand description

§Credential pools

One account’s quota must never dead-end a turn. A pool is N providers of one kind built from N credentials, one account each. Handed to a FallbackProvider in order, they are ordinary tiers: account 1 serves every turn until it refuses on quota (ErrorKind::RateLimit — a spent subscription window, a 429, an HTTP API’s exhausted prepaid credit), the chain moves to account 2, and the refused account is held in cooldown until it is worth asking again.

provider_pool is the provider-agnostic builder: it takes accounts the caller already built — an HTTP provider per API key, such as one AnthropicProvider per AnthropicConfig::pool_from_env entry (feature http-api) — and names them. cli_pool builds the accounts of a CLI runner type itself, each carrying its own token as explicit child environment (RunnerConfig::with_env), so the sandbox’s clearing of the process environment never matters and every runner logs in as its own account.

Attribution stays per account. Tier N is named <provider>#<N> (the first keeps the provider’s own name), so a health snapshot, a quota alert label or a usage row keyed on the serving tier’s name says which account served. A response keeps the model the account actually ran (claude-haiku-5-5, never anthropic#2): pricing looks the pair up as (tier name, model), keys the tier on the family before the # (crate::pricing::pool_family) and the model on its id prefix, so every account bills as its provider.

There is no load spreading. Spreading turns across accounts would hide a spent account until the last one dies, which is the failure this exists to remove.

Structs§

PooledTier
One account of a pool: a runner under a per-account name.

Functions§

cli_pool
Build one tier per credential, in the order given.
credential_env_key
The variable a runner type reads its credential from, when it reads one.
provider_pool
Name already-built accounts as the tiers of one pool, in the order given.