Struct CompatibleProvider 

pub struct CompatibleProvider { /* private fields */ }

LlmProvider adapter for OpenAI-compatible REST endpoints.

Delegates all operations to an inner OpenAiProvider while exposing a configurable provider_name for logging and routing identification.

Implementations

impl CompatibleProvider

pub fn new(cfg: CompatibleConfig) -> Self

Create a new provider from a CompatibleConfig.

impl CompatibleProvider

pub async fn list_models_remote(&self) -> Result<Vec<RemoteModelInfo>, LlmError>

Fetch models via the inner OpenAiProvider. Cache slug is derived from base URL.

Errors

Returns an error if the API request fails.

impl CompatibleProvider

pub fn set_status_tx(&mut self, tx: StatusTx)

Attach a status channel for streaming progress events to the TUI.

pub fn with_generation_overrides(self, overrides: GenerationOverrides) -> Self

Override generation parameters (temperature, top-p, etc.) for all subsequent calls.

pub fn with_completion_tokens_param(self, param: CompletionTokensParam) -> Self

Override which token-limit parameter is sent in API requests.

Delegates to the inner OpenAiProvider. Use this when the model name is not covered by the built-in prefix table and the inferred field would produce a 400 error.

Examples

use zeph_llm::compatible::{CompatibleConfig, CompatibleProvider};
use zeph_llm::openai::CompletionTokensParam;

let provider = CompatibleProvider::new(CompatibleConfig {
    provider_name: "my-provider".into(),
    api_key: "key".into(),
    base_url: "https://api.example.com/v1".into(),
    model: "my-ft-reasoner-v1".into(),
    max_tokens: 4096,
    embedding_model: None,
    completion_tokens_param: None,
    vision: None,
})
.with_completion_tokens_param(CompletionTokensParam::MaxCompletionTokens);

pub fn with_vision(self, supported: bool) -> Self

Override the vision-capability value reported by LlmProvider::supports_vision.

Delegates to the inner OpenAiProvider. Use this for compatible endpoints whose model name is not covered by OpenAiProvider’s built-in prefix table — which is the common case, since third-party model names carry no OpenAI naming convention and the table therefore fails safe to false for them.

Examples

use zeph_llm::compatible::{CompatibleConfig, CompatibleProvider};
use zeph_llm::provider::LlmProvider;

let provider = CompatibleProvider::new(CompatibleConfig {
    provider_name: "local-vlm".into(),
    api_key: "key".into(),
    base_url: "http://localhost:8000/v1".into(),
    model: "llava-onevision".into(),
    max_tokens: 4096,
    embedding_model: None,
    completion_tokens_param: None,
    vision: None,
})
.with_vision(true);
assert!(provider.supports_vision());

pub fn with_output_schema_forwarding( self, enabled: bool, hint_bytes: usize, max_description_bytes: usize, ) -> Self

Forward MCP tool output schemas as JSON hints appended to tool descriptions.

Delegates to the inner OpenAiProvider. When enabled is false the call is a no-op. hint_bytes caps the JSON representation; max_description_bytes caps the combined description string.

pub fn set_reasoning_effort(&mut self, effort: Option<String>)

Apply a reasoning_effort override to the inner OpenAiProvider.

Delegates to OpenAiProvider::set_reasoning_effort, which validates the value ("low", "medium", or "high") and logs a warning for any unknown value. Pass None to clear a previously-set effort level.

pub fn current_reasoning_effort(&self) -> Option<String>

Return the currently configured reasoning_effort value on the inner OpenAiProvider, if any.

Trait Implementations

impl Clone for CompatibleProvider

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CompatibleProvider

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LlmProvider for CompatibleProvider

fn context_window(&self) -> Option<usize>

Report the model’s context window size in tokens.

async fn chat(&self, messages: &[Message]) -> Result<String, LlmError>

Send messages to the LLM and return the assistant response.

async fn chat_with_extras( &self, messages: &[Message], ) -> Result<(String, ChatExtras), LlmError>

Send messages and return the assistant response together with per-call extras.

async fn chat_stream( &self, messages: &[Message], ) -> Result<ChatStream, LlmError>

Send messages and return a stream of response chunks.

fn supports_streaming(&self) -> bool

Whether this provider supports native streaming.

async fn embed(&self, text: &str) -> Result<Vec<f32>, LlmError>

Generate an embedding vector from text.

async fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, LlmError>

Embed multiple texts in a single API call.

fn supports_embeddings(&self) -> bool

Whether this provider supports embedding generation.

fn name(&self) -> &str

Provider name for logging and identification.

fn model_identifier(&self) -> &str

Model identifier string (e.g. gpt-4o-mini, claude-sonnet-5). Used by cost-estimation heuristics. Returns "" when not applicable.

fn list_models(&self) -> Vec<String>

Return the list of model identifiers this provider can serve. Default: empty (provider does not advertise models).

fn supports_structured_output(&self) -> bool

Whether this provider supports native structured output.

async fn chat_typed<T>(&self, messages: &[Message]) -> Result<T, LlmError>

where T: DeserializeOwned + JsonSchema + 'static, Self: Sized,

Send messages and parse the response into a typed value T.

async fn chat_with_tools( &self, messages: &[Message], tools: &[ToolDefinition], ) -> Result<ChatResponse, LlmError>

Send messages with tool definitions, returning a structured response.

fn last_cache_usage(&self) -> Option<(u64, u64)>

Return the cache usage from the last API call, if available. Returns (cache_creation_tokens, cache_read_tokens).

fn last_usage(&self) -> Option<(u64, u64)>

Return token counts from the last API call, if available. Returns (input_tokens, output_tokens).

fn last_reasoning_tokens(&self) -> Option<u64>

Return reasoning tokens from the last API call, if the provider reports them.

fn supports_vision(&self) -> bool

Whether this provider supports image input (vision).

fn supports_tool_use(&self) -> bool

Whether this provider supports native tool_use / function calling.

fn debug_request_json( &self, messages: &[Message], tools: &[ToolDefinition], stream: bool, ) -> Value

Return the request payload that will be sent to the provider, for debug dumps.

fn effective_model_identifier(&self) -> &str

Model identifier that actually served the most recent dispatch.

fn last_ttft_ms(&self) -> Option<u64>

Return the time-to-first-byte (milliseconds) of the last API call, if available.

fn take_compaction_summary(&self) -> Option<String>

Return the compaction summary from the most recent API call, if a server-side compaction occurred (Claude compact-2026-01-12 beta). Clears the stored value.