pub struct Client { /* private fields */ }Expand description
Async client for the llmleaf gateway.
Construct with Client::new for the common case, or Client::builder to set a
timeout, an admin token, or supply your own reqwest::Client. The client is cheap
to clone (it wraps an Arc-backed reqwest::Client).
Implementations§
Source§impl Client
impl Client
Sourcepub fn new(
base_url: impl Into<String>,
api_key: impl Into<String>,
) -> Result<Self>
pub fn new( base_url: impl Into<String>, api_key: impl Into<String>, ) -> Result<Self>
Construct a client from a base URL and API key with default settings.
Sourcepub fn builder(
base_url: impl Into<String>,
api_key: impl Into<String>,
) -> ClientBuilder
pub fn builder( base_url: impl Into<String>, api_key: impl Into<String>, ) -> ClientBuilder
Start a ClientBuilder.
Sourcepub async fn chat(&self, request: ChatRequest) -> Result<ChatResponse>
pub async fn chat(&self, request: ChatRequest) -> Result<ChatResponse>
POST /v1/chat/completions (non-streaming).
stream is forced to false (or absent) so a JSON ChatResponse comes back.
Sourcepub async fn chat_stream(
&self,
request: ChatRequest,
) -> Result<impl Stream<Item = Result<ChatCompletionChunk>>>
pub async fn chat_stream( &self, request: ChatRequest, ) -> Result<impl Stream<Item = Result<ChatCompletionChunk>>>
POST /v1/chat/completions (streaming, SSE).
Forces stream:true, then yields decoded ChatCompletionChunks, stopping on the
data: [DONE] sentinel (which is never parsed). Accumulate
choices[].delta.content for the assembled text.
Sourcepub async fn responses(
&self,
request: ResponsesRequest,
) -> Result<ResponsesResponse>
pub async fn responses( &self, request: ResponsesRequest, ) -> Result<ResponsesResponse>
POST /v1/responses (non-streaming) — the OpenAI Responses dialect.
stream is forced to false (or absent) so a JSON ResponsesResponse comes
back. llmleaf serves this dialect statelessly, so the response always reports
"store": false (SPEC.md).
Sourcepub async fn responses_stream(
&self,
request: ResponsesRequest,
) -> Result<impl Stream<Item = Result<ResponsesStreamEvent>>>
pub async fn responses_stream( &self, request: ResponsesRequest, ) -> Result<impl Stream<Item = Result<ResponsesStreamEvent>>>
POST /v1/responses (streaming, typed SSE).
Forces stream:true, then yields decoded ResponsesStreamEvents. Unlike chat
there is no [DONE] sentinel: the stream ends after the terminal
response.completed / response.incomplete / response.failed event. Unrecognised
event types are skipped; the "error" event surfaces as an Error::Api, the same
way the chat stream surfaces a mid-stream failure. Accumulate
ResponsesStreamEvent::output_text_delta for the assembled text.
Sourcepub async fn embeddings(
&self,
request: EmbeddingRequest,
) -> Result<EmbeddingResponse>
pub async fn embeddings( &self, request: EmbeddingRequest, ) -> Result<EmbeddingResponse>
POST /v1/embeddings. Base64 payloads (encoding_format:"base64") are decoded
into float vectors before returning.
Sourcepub async fn rerank(&self, request: RerankRequest) -> Result<RerankResponse>
pub async fn rerank(&self, request: RerankRequest) -> Result<RerankResponse>
POST /v1/rerank. Scores each of request.documents against request.query
and returns the results ordered by relevance. documents may be plain strings or
structured multimodal objects; when return_documents is set the originals are
echoed back on each RerankResult.
Sourcepub async fn list_models(
&self,
model_type: Option<ModelType>,
search: Option<&str>,
) -> Result<ListModelsResponse>
pub async fn list_models( &self, model_type: Option<ModelType>, search: Option<&str>, ) -> Result<ListModelsResponse>
GET /v1/models. type filters the catalog; search is a substring match. The
per-model endpoints array appears only when the client was built with an admin
token.
Sourcepub async fn speech(&self, request: SpeechRequest) -> Result<(Bytes, String)>
pub async fn speech(&self, request: SpeechRequest) -> Result<(Bytes, String)>
POST /v1/audio/speech. Returns the raw audio bytes and the resolved
Content-Type (from the response header, falling back to the SPEC.md table for the
requested response_format).
Sourcepub async fn voices(&self, model: &str) -> Result<VoicesResponse>
pub async fn voices(&self, model: &str) -> Result<VoicesResponse>
GET /v1/audio/voices?model=<id>.
Sourcepub async fn transcribe(
&self,
request: TranscriptionRequest,
file_name: impl Into<String>,
audio: impl Into<Bytes>,
) -> Result<Transcription>
pub async fn transcribe( &self, request: TranscriptionRequest, file_name: impl Into<String>, audio: impl Into<Bytes>, ) -> Result<Transcription>
POST /v1/audio/transcriptions (multipart). file carries the audio bytes under
file_name. Returns a structured Transcription::Json for json/verbose_json, or
a plain-text Transcription::Text for text/srt/vtt (SPEC.md).
Sourcepub async fn create_batch(
&self,
request: BatchCreateRequest,
) -> Result<BatchHandle>
pub async fn create_batch( &self, request: BatchCreateRequest, ) -> Result<BatchHandle>
POST /v1/batches.
Sourcepub async fn get_batch(&self, id: &str) -> Result<BatchHandle>
pub async fn get_batch(&self, id: &str) -> Result<BatchHandle>
GET /v1/batches/{id}.
Sourcepub async fn cancel_batch(&self, id: &str) -> Result<BatchHandle>
pub async fn cancel_batch(&self, id: &str) -> Result<BatchHandle>
POST /v1/batches/{id}/cancel.
Sourcepub async fn batch_results(
&self,
id: &str,
) -> Result<impl Stream<Item = Result<BatchResultLine>>>
pub async fn batch_results( &self, id: &str, ) -> Result<impl Stream<Item = Result<BatchResultLine>>>
GET /v1/batches/{id}/results (application/x-ndjson). Yields one
BatchResultLine per line.