pub trait OpenAICompatibleProvider: Provider {
type StreamingUsage: Clone + Default + GetTokenUsage + Serialize + DeserializeOwned + Unpin + WasmCompatSend + WasmCompatSync + 'static;
type Response: DeserializeOwned + Serialize + ProviderResponseExt + TryInto<CompletionResponse<Self::Response>, Error = CompletionError> + WasmCompatSend + WasmCompatSync
where <Self::Response as ProviderResponseExt>::Usage: GetTokenUsage;
const PROVIDER_NAME: &'static str;
const EMITS_COMPLETE_SINGLE_CHUNK_TOOL_CALLS: bool = false;
const SUPPORTS_TOOLS: bool = true;
const SUPPORTS_RESPONSE_FORMAT: bool = true;
const STREAM_INCLUDE_USAGE: bool = true;
// Provided methods
fn completion_path(&self, model: &str) -> String { ... }
fn build_completion_request(
&self,
model: String,
request: CompletionRequest,
options: CompletionModelOptions,
) -> Result<CompletionRequest, CompletionError> { ... }
fn prepare_request(
&self,
request: &mut CompletionRequest,
) -> Result<(), CompletionError> { ... }
fn finalize_request_body(
&self,
body: &mut Value,
) -> Result<(), CompletionError> { ... }
fn finalize_request_body_with_options(
&self,
body: &mut Value,
options: CompletionModelOptions,
) -> Result<(), CompletionError> { ... }
fn decorate_streaming_tool_call(
&self,
detail: &Value,
tool_calls: &mut HashMap<usize, RawStreamingToolCall>,
) { ... }
}Expand description
Contract for provider extensions that speak the OpenAI Chat Completions wire
format through GenericCompletionModel. Mirrors
AnthropicCompatibleProvider
on the Anthropic-compatible side.
Request construction runs the hooks in a fixed order:
prepare_request on the typed request, then
serialization, then (for streaming) the stream/stream_options merge,
and finally
finalize_request_body_with_options
on the serialized body — so the finalize hook always sees the streaming
parameters and model-level options.
Required Associated Constants§
Sourceconst PROVIDER_NAME: &'static str
const PROVIDER_NAME: &'static str
Provider name recorded on gen_ai.provider.name telemetry spans.
Provided Associated Constants§
Sourceconst EMITS_COMPLETE_SINGLE_CHUNK_TOOL_CALLS: bool = false
const EMITS_COMPLETE_SINGLE_CHUNK_TOOL_CALLS: bool = false
Whether the backend can emit a whole tool call (id, name, and complete arguments) in a single streaming chunk, as llama.cpp-based servers do. When true, the shared streaming layer emits such calls as soon as they arrive instead of holding them until the stream ends.
Sourceconst SUPPORTS_TOOLS: bool = true
const SUPPORTS_TOOLS: bool = true
Whether the provider supports tool calling. When false, tools and
tool_choice are dropped with a warning during request conversion —
before tool-choice validation, so unsupported tool configurations
never error client-side on a provider that ignores tools anyway.
Sourceconst SUPPORTS_RESPONSE_FORMAT: bool = true
const SUPPORTS_RESPONSE_FORMAT: bool = true
Whether output_schema maps to OpenAI’s response_format. Providers
whose APIs reject json_schema response formats set this to false;
the schema is then dropped with a warning instead of being sent.
Sourceconst STREAM_INCLUDE_USAGE: bool = true
const STREAM_INCLUDE_USAGE: bool = true
Whether streaming requests include
"stream_options": {"include_usage": true}. Providers that reject
unknown parameters and already report usage on the final chunk set
this to false.
Required Associated Types§
Sourcetype StreamingUsage: Clone + Default + GetTokenUsage + Serialize + DeserializeOwned + Unpin + WasmCompatSend + WasmCompatSync + 'static
type StreamingUsage: Clone + Default + GetTokenUsage + Serialize + DeserializeOwned + Unpin + WasmCompatSend + WasmCompatSync + 'static
The usage payload parsed from streaming chunks and carried on the
final streaming response. OpenAI’s Usage for most providers;
providers with richer usage accounting (e.g. Mistral’s cached-token
fallbacks, DeepSeek’s cache hit/miss counters) substitute their own.
Sourcetype Response: DeserializeOwned + Serialize + ProviderResponseExt + TryInto<CompletionResponse<Self::Response>, Error = CompletionError> + WasmCompatSend + WasmCompatSync
where
<Self::Response as ProviderResponseExt>::Usage: GetTokenUsage
type Response: DeserializeOwned + Serialize + ProviderResponseExt + TryInto<CompletionResponse<Self::Response>, Error = CompletionError> + WasmCompatSend + WasmCompatSync where <Self::Response as ProviderResponseExt>::Usage: GetTokenUsage
The chat-completions payload this provider returns.
Provided Methods§
Sourcefn completion_path(&self, model: &str) -> String
fn completion_path(&self, model: &str) -> String
The request path for chat completions, resolved against the client
base URL by Provider::build_uri.
Providers that route the model through the URL (e.g. Azure deployment
paths) or keep other capabilities on differently-versioned paths
override this. model is the identifier the completion model handle
was created with; per-request model overrides only affect the body.
Sourcefn build_completion_request(
&self,
model: String,
request: CompletionRequest,
options: CompletionModelOptions,
) -> Result<CompletionRequest, CompletionError>
fn build_completion_request( &self, model: String, request: CompletionRequest, options: CompletionModelOptions, ) -> Result<CompletionRequest, CompletionError>
Build the typed chat-completions request. Providers that share the
OpenAI transport but need provider-specific message conversion can
override this while still using GenericCompletionModel for sending,
streaming, error handling, and telemetry.
Sourcefn prepare_request(
&self,
request: &mut CompletionRequest,
) -> Result<(), CompletionError>
fn prepare_request( &self, request: &mut CompletionRequest, ) -> Result<(), CompletionError>
Adjust the typed request before serialization (e.g. rewrite the model
identifier or fold provider-native tool definitions out of
additional_params).
Sourcefn finalize_request_body(&self, body: &mut Value) -> Result<(), CompletionError>
fn finalize_request_body(&self, body: &mut Value) -> Result<(), CompletionError>
Adjust the fully serialized request body — after any streaming
parameters are merged — immediately before it is sent. This is where
wire-level dialect differences live (e.g. Mistral’s "any" tool
choice, DeepSeek’s string-flattened message content).
Sourcefn finalize_request_body_with_options(
&self,
body: &mut Value,
options: CompletionModelOptions,
) -> Result<(), CompletionError>
fn finalize_request_body_with_options( &self, body: &mut Value, options: CompletionModelOptions, ) -> Result<(), CompletionError>
Adjust the fully serialized request body with model-level options.
Providers that do not need model-instance options should override
finalize_request_body instead.
Sourcefn decorate_streaming_tool_call(
&self,
detail: &Value,
tool_calls: &mut HashMap<usize, RawStreamingToolCall>,
)
fn decorate_streaming_tool_call( &self, detail: &Value, tool_calls: &mut HashMap<usize, RawStreamingToolCall>, )
Decorate streamed tool calls from provider-specific streaming detail payloads. Most OpenAI-compatible providers do not emit such details.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".