Struct Client 

pub struct Client { /* private fields */ }

A handle to one SIE server.

Cloning is cheap and shares the underlying connection pool, so a Client can be stored once and used from every task.

Implementations

impl Client

pub fn batches(&self) -> Batches

Operations on offline batches.

impl Client

pub fn connections(&self) -> Connections

Operations on stored data-store credentials.

impl Client

pub fn encode( &self, model: impl Into<String>, items: impl IntoIterator<Item = Item>, ) -> EncodeRequest

Embed one or more items.

pub fn score( &self, model: impl Into<String>, query: Item, items: impl IntoIterator<Item = Item>, ) -> ScoreRequest

Score a query against a set of candidates.

pub fn extract( &self, model: impl Into<String>, items: impl IntoIterator<Item = Item>, ) -> ExtractRequest

Extract structure from one or more items.

impl Client

pub fn files(&self) -> Files

Operations on uploaded files.

impl Client

pub async fn estimate( &self, endpoint: &str, request: Value, ) -> Result<CostEstimate>

Estimate the cost of a request before sending it.

endpoint is the absolute path the request would be sent to, and request is the body it would carry.

pub fn generate( &self, model: impl Into<String>, prompt: impl Into<String>, max_new_tokens: u32, ) -> GenerateRequest

Generate text with the native endpoint.

pub fn chat( &self, model: impl Into<String>, messages: impl IntoIterator<Item = ChatMessage>, ) -> ChatRequest

Generate a chat completion.

pub fn responses( &self, model: impl Into<String>, input: ResponseInput, ) -> ResponsesRequest

Generate a response through the Responses API.

impl Client

pub fn jobs(&self) -> Jobs

Operations on batch jobs.

impl Client

pub async fn list_models(&self) -> Result<Vec<ModelInfo>>

Every model the server knows about.

pub async fn get_model(&self, model: &str) -> Result<ModelInfo>

Details for one model.

pub async fn health(&self) -> Result<HealthResponse>

Health, from either a gateway or a standalone worker.

A gateway answers /health with the whole cluster’s state. A worker running on its own has no such view: it serves the Kubernetes-style /healthz probe, which is plain text, so its reply is reported as a worker with that status and nothing else.

pub async fn get_capacity(&self, gpu: Option<&str>) -> Result<CapacityInfo>

Cluster capacity, optionally narrowed to one GPU type.

Requires a gateway: a worker’s /health describes only itself.

pub async fn wait_for_capacity( &self, gpu: &str, model: Option<&str>, timeout: Option<Duration>, poll_interval: Duration, ) -> Result<CapacityInfo>

Block until a GPU type has capacity.

With model, this warms the model as well: it issues one small encode with capacity waiting enabled, which triggers both scale-up and model loading, then reports the resulting capacity. Without it, /health is polled until a worker appears.

impl Client

pub fn pools(&self) -> Pools

Operations on worker pools.

impl Client

pub async fn watch(&self, mode: WatchMode) -> Result<ChunkStream<StatusMessage>>

Stream status broadcasts until the connection closes.

With WatchMode::Auto, /health is probed once to decide which endpoint to open.

impl Client

pub fn new(base_url: impl AsRef<str>) -> Result<Self>

A client with default settings.

pub fn builder(base_url: impl AsRef<str>) -> ClientBuilder

Start configuring a client.

pub fn base_url(&self) -> &str

The server root, always with a trailing slash.

pub fn default_options(&self) -> &RequestOptions

Per-call defaults this client applies when a request builder does not override them.

Trait Implementations

impl Clone for Client

fn clone(&self) -> Client

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Client

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.