Struct Client 

pub struct Client<CTXEXT, FENSLLM, CUSG, FENS, FVVOTE, FCVOTE, VUSG> {
    pub chat_client: Arc<Client<CTXEXT, FENSLLM, CUSG>>,
    pub ensemble_fetcher: Arc<CachingFetcher<CTXEXT, FENS>>,
    pub completion_votes_fetcher: Arc<FVVOTE>,
    pub cache_vote_fetcher: Arc<FCVOTE>,
    pub usage_handler: Arc<VUSG>,
}

Client for creating vector completions.

Orchestrates multiple LLM chat completions to vote on response options, combining their votes using weights to produce final scores.

Fields

chat_client: Arc<Client<CTXEXT, FENSLLM, CUSG>>

The underlying chat completion client.

ensemble_fetcher: Arc<CachingFetcher<CTXEXT, FENS>>

Fetcher for Ensemble definitions.

completion_votes_fetcher: Arc<FVVOTE>

Fetcher for votes from historical completions.

cache_vote_fetcher: Arc<FCVOTE>

Fetcher for votes from the global cache.

usage_handler: Arc<VUSG>

Handler for usage tracking.

Implementations

impl<CTXEXT, FENSLLM, CUSG, FENS, FVVOTE, FCVOTE, VUSG> Client<CTXEXT, FENSLLM, CUSG, FENS, FVVOTE, FCVOTE, VUSG>

pub fn new( chat_client: Arc<Client<CTXEXT, FENSLLM, CUSG>>, ensemble_fetcher: Arc<CachingFetcher<CTXEXT, FENS>>, completion_votes_fetcher: Arc<FVVOTE>, cache_vote_fetcher: Arc<FCVOTE>, usage_handler: Arc<VUSG>, ) -> Self

Creates a new vector completion client.

impl<CTXEXT, FENSLLM, CUSG, FENS, FVVOTE, FCVOTE, VUSG> Client<CTXEXT, FENSLLM, CUSG, FENS, FVVOTE, FCVOTE, VUSG>

where CTXEXT: ContextExt + Send + Sync + 'static, FENSLLM: Fetcher<CTXEXT> + Send + Sync + 'static, CUSG: UsageHandler<CTXEXT> + Send + Sync + 'static, FENS: Fetcher<CTXEXT> + Send + Sync + 'static, FVVOTE: Fetcher<CTXEXT> + Send + Sync + 'static, FCVOTE: Fetcher<CTXEXT> + Send + Sync + 'static, VUSG: UsageHandler<CTXEXT> + Send + Sync + 'static,

pub async fn create_unary_handle_usage( self: Arc<Self>, ctx: Context<CTXEXT>, request: Arc<VectorCompletionCreateParams>, ) -> Result<VectorCompletion, Error>

Creates a unary (non-streaming) vector completion with usage tracking.

Collects all streaming chunks into a single response.

pub async fn create_streaming_handle_usage( self: Arc<Self>, ctx: Context<CTXEXT>, request: Arc<VectorCompletionCreateParams>, ) -> Result<impl Stream<Item = VectorCompletionChunk> + Send + Unpin + 'static, Error>

Creates a streaming vector completion with usage tracking.

Spawns a background task to track usage after the stream completes.

impl<CTXEXT, FENSLLM, CUSG, FENS, FVVOTE, FCVOTE, VUSG> Client<CTXEXT, FENSLLM, CUSG, FENS, FVVOTE, FCVOTE, VUSG>

where CTXEXT: ContextExt + Send + Sync + 'static, FENSLLM: Fetcher<CTXEXT> + Send + Sync + 'static, CUSG: UsageHandler<CTXEXT> + Send + Sync + 'static, FENS: Fetcher<CTXEXT> + Send + Sync + 'static, FVVOTE: Fetcher<CTXEXT> + Send + Sync + 'static, FCVOTE: Fetcher<CTXEXT> + Send + Sync + 'static, VUSG: Send + Sync + 'static,

pub async fn create_streaming( self: Arc<Self>, ctx: Context<CTXEXT>, request: Arc<VectorCompletionCreateParams>, ) -> Result<impl Stream<Item = VectorCompletionChunk> + Send + 'static, Error>

Creates a streaming vector completion.

Orchestrates chat completions across all LLMs in the ensemble, extracting votes from each and combining them with weights to produce scores.