Crate iron_providers 

iron-providers: Semantic provider boundary for protocol-oriented LLM providers

This crate provides a function-oriented provider API that normalizes inference requests and responses across different LLM backends. Supports Responses, Chat Completions, and Messages protocol families via a profile-driven provider connection and registry.

Architecture

The crate is organized around a few core concepts:

• Provider trait: The async interface for performing inference. Callers use Provider::infer for non-streaming and Provider::infer_stream for streaming requests.

• InferenceRequest: The normalized request type carrying model ID, instructions, tools, and an InferenceContext.

• InferenceContext: Separates model-visible conversation (Transcript) from runtime-only records (RuntimeRecord). Provider adapters project only the transcript into model-visible request fields. Runtime records may influence request assembly through explicit provider-specific mapping logic but are never replayed as assistant text.

• ProviderEvent: Normalized streaming events. The stream termination contract is:

  • ProviderEvent::Complete is emitted only on successful stream termination.
  • If a provider encounters an unrecoverable error, the stream ends with ProviderEvent::Error and does not emit Complete.

• ProviderProfile: Declarative provider configuration including API family, base URL, auth strategy, default headers, and quirks. All provider families (Responses, Chat Completions, Messages) honor the full profile model consistently.

• ProviderConnection: Resolved provider state that implements Provider. Construct directly from a ProviderProfile and RuntimeConfig, or obtain from ProviderRegistry::get.

• ProviderRegistry: Registry for looking up providers by slug or URL pattern, with built-in profiles for common providers. Includes local for OpenAI-compatible local endpoints (Ollama, LM Studio, etc.) via /v1/chat/completions, ollama-cloud for the Ollama cloud API, and openai for public OpenAI Responses API-key access.

Streaming

Streaming is determined by which method you call (infer vs infer_stream), not by a field on the request. There is no stream field on InferenceRequest.

Re-exports

pub use connection::ProviderConnection;

pub use error::ProviderError;

pub use error::ProviderResult;

pub use model::ChoiceItem;

pub use model::ChoiceRequest;

pub use model::ChoiceSelectionMode;

pub use model::GenerationConfig;

pub use model::InferenceContext;

pub use model::InferenceRequest;

pub use model::Message;

pub use model::ProviderEvent;

pub use model::RuntimeRecord;

pub use model::TokenUsage;

pub use model::ToolCall;

pub use model::ToolDefinition;

pub use model::ToolPolicy;

pub use model::Transcript;

pub use model::CHOICE_REQUEST_TOOL_NAME;

pub use profile::ApiFamily;

pub use profile::AuthStrategy;

pub use profile::CredentialAuthConfig;

pub use profile::CredentialKind;

pub use profile::EndpointPurpose;

pub use profile::ProviderCredential;

pub use profile::ProviderProfile;

pub use profile::ProviderQuirks;

pub use profile::RuntimeConfig;

pub use profile::RuntimeConfigSource;

pub use provider::Provider;

pub use provider::ProviderFuture;

pub use registry::ProviderRegistry;

Modules

connection

Provider connection — resolved provider state

error

Provider error types

model

Semantic provider request/response models

prelude

profile

provider

Provider trait definition.

registry