Struct HooshClient 

pub struct HooshClient { /* private fields */ }

HTTP client for the hoosh inference gateway.

Implementations

impl HooshClient

pub fn new(base_url: impl Into<String>) -> Self

Create a new client pointing at the given hoosh server.

The client is tuned for low-latency local connections:

• TCP_NODELAY disables Nagle’s algorithm (avoids 40ms batching delay)
• Connection pooling keeps TCP connections alive across requests
• HTTP/2 adaptive window for multiplexed requests

pub async fn infer( &self, request: &InferenceRequest, ) -> Result<InferenceResponse>

Run inference via the hoosh server.

pub async fn infer_stream( &self, request: &InferenceRequest, ) -> Result<Receiver<Result<String, HooshError>>>

Stream inference results token by token.

pub async fn list_models(&self) -> Result<Vec<ModelInfo>>

List available models.

pub async fn health(&self) -> Result<bool>

Health check.

pub fn base_url(&self) -> &str

Base URL of the hoosh server.

Trait Implementations

impl Clone for HooshClient

fn clone(&self) -> HooshClient

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HooshClient

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.