Struct OpenAiCompatible 

pub struct OpenAiCompatible { /* private fields */ }

Generic OpenAI-compatible model provider. No default base URL — always construct with OpenAiCompatible::new.

Implementations

impl OpenAiCompatible

pub fn new(base_url: impl Into<String>) -> Self

Create a client targeting the given base URL (e.g. http://localhost:8000).

pub fn with_base_url(self, url: impl Into<String>) -> Self

Override the base URL set at construction.

pub fn with_model(self, name: impl Into<String>) -> Self

Set the model name sent in the request body.

pub fn with_api_key(self, key: impl Into<String>) -> Self

Set the API key / bearer token, if the server requires one.

pub fn with_timeout(self, timeout: Duration) -> Self

Set a custom timeout for HTTP requests.

pub fn with_max_retries(self, retries: u32) -> Self

Set the maximum number of retries for transient (429 / 5xx) errors on the initial request (default: 3).

pub fn allow_plaintext_api_key(self) -> Self

Opts back into warn-and-send for an API key sent over a plaintext http:// base URL (default: hard error). Only use this for a genuinely local, unauthenticated-but-keyed server.

pub fn with_extra_field(self, key: impl Into<String>, value: Value) -> Self

Add an arbitrary top-level field to every chat request body, for server-specific sampling parameters this generic client doesn’t know about by name (e.g. vLLM’s repetition_penalty).

Trait Implementations

impl Debug for OpenAiCompatible

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Model for OpenAiCompatible

async fn generate(&self, request: &ChatRequest) -> Result<ChatResponse>

Generates a complete response for the given request.

async fn generate_stream(&self, request: &ChatRequest) -> Result<ResponseStream>

Returns a stream of response chunks for token-by-token output.

fn model_id(&self) -> &str

The provider-side model identifier (e.g. "claude-sonnet-5"), used for cost attribution. Defaults to "default" so existing implementations keep compiling; providers should override it with the configured model name.