Struct HyperInferClient 

pub struct HyperInferClient { /* private fields */ }

Implementations

impl HyperInferClient

pub async fn new( redis_url: &str, config: Config, ) -> Result<Self, HyperInferError>

pub async fn set_mirror(&self, cfg: Option<MirrorConfig>)

Configure traffic mirroring. Pass None to disable.

pub async fn inject_provider_registry( &self, external_registry: Arc<ProviderRegistry>, )

pub async fn load_deployments(&self, deployments: Vec<Deployment>)

Load deployments into the router engine for deployment-based routing

pub async fn subscribe_config_updates<F, Fut>( &self, redis_url: &str, fetcher: F, ) -> Result<(), HyperInferError>

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = Result<Vec<Deployment>, String>> + Send,

Subscribe to Redis Pub/Sub for live deployment config changes. When a message arrives on “hyperinfer:config_updates”, calls the provided fetcher function to re-fetch deployments and rebuilds the routing pool.

pub fn router_engine(&self) -> &Arc<RouterEngine>

Get a reference to the router engine

pub async fn chat( &self, key: &str, request: ChatRequest, ) -> Result<ChatResponse, HyperInferError>

pub async fn chat_stream( &self, key: &str, request: ChatRequest, ) -> Result<Pin<Box<dyn Stream<Item = Result<ChatChunk, HyperInferError>> + Send>>, HyperInferError>

Stream token chunks for a chat request.

Returns a Stream of ChatChunk items. The caller is responsible for collecting delta fields and assembling the final text. The last chunk in the stream has a non-None finish_reason and may carry usage.

Rate-limiting and routing follow the same logic as chat().