Trait UsageHandler 

pub trait UsageHandler<CTXEXT> {
    // Required method
    fn handle_usage<'life0, 'async_trait>(
        &'life0 self,
        ctx: Context<CTXEXT>,
        request: Option<Arc<ChatCompletionCreateParams>>,
        response: ChatCompletion,
    ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
}

Trait for handling usage tracking after chat completions.

Required Methods

fn handle_usage<'life0, 'async_trait>( &'life0 self, ctx: Context<CTXEXT>, request: Option<Arc<ChatCompletionCreateParams>>, response: ChatCompletion, ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Called after a chat completion finishes to record usage.

The request is None when called from vector completions.

Implementors

impl<CTXEXT> UsageHandler<CTXEXT> for LogUsageHandler

where CTXEXT: Send + Sync + 'static,