Module client 

HTTP clients for streaming LLM responses with performance recording

This module provides HTTP clients that leverage async-openai with BYOT (Bring Your Own Types) feature to work with OpenAI-compatible APIs. The clients support recording streaming responses for performance analysis.

Structs

BaseHttpClient

Base HTTP client with common functionality

GenericBYOTClient

Generic BYOT client using serde_json::Value for maximum flexibility

HttpClientConfig

Configuration for HTTP clients

HttpRequestContext

Context for HTTP client requests that supports cancellation This bridges AsyncEngineContext and reqwest cancellation

HttpResponseStream

A wrapped HTTP response stream that combines a stream with its context This provides a unified interface for HTTP client responses

NvCustomClient

NV Custom client using NvCreateChatCompletionRequest with Annotated responses

PureOpenAIClient

Pure OpenAI client using standard async-openai types

Enums

HttpClientError

Error types for HTTP clients

Type Aliases

ByotHttpResponseStream

Type alias for HttpResponseStream with BYOT responses

ByotResponseStream

Type alias for generic BYOT response stream

NvChatResponseStream

Type alias for NV chat response stream

NvHttpResponseStream

Type alias for HttpResponseStream with NV chat completion responses

OpenAIChatResponseStream

Type alias for pure OpenAI chat response stream

OpenAIHttpResponseStream

Type alias for HttpResponseStream with pure OpenAI responses