Skip to main content

LLMClient

llm_kernel::llm::client

Trait LLMClient 

Source 
pub trait LLMClient: Send + Sync {
    // Required methods
    fn complete<'life0, 'async_trait>(
        &'life0 self,
        request: LLMRequest,
    ) -> Pin<Box<dyn Future<Output = Result<LLMResponse>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
    fn model_name(&self) -> &str;
    fn stream_complete<'life0, 'async_trait>(
        &'life0 self,
        request: LLMRequest,
    ) -> Pin<Box<dyn Future<Output = Result<LLMStream>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
}
Expand description

Unified async interface for LLM chat completion and streaming.

Required Methods§

Source

fn complete<'life0, 'async_trait>( &'life0 self, request: LLMRequest, ) -> Pin<Box<dyn Future<Output = Result<LLMResponse>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Send a chat completion request and return the full response.

Reasoning-model answer promotion (non-streaming only): when a provider returns the final answer in reasoning_content and leaves content empty (notably GLM-4.7), complete promotes the reasoning into content so downstream consumers transparently receive the answer. The original reasoning is preserved in LLMResponse::reasoning.

This promotion is not applied by stream_complete, which surfaces reasoning as separate StreamEvent::ReasoningDelta events — see that variant’s docs for the streaming accumulation contract.

Source

fn model_name(&self) -> &str

Return the model name this client is configured to use.

Source

fn stream_complete<'life0, 'async_trait>( &'life0 self, request: LLMRequest, ) -> Pin<Box<dyn Future<Output = Result<LLMStream>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Stream a chat completion, yielding events as they arrive.

Does not promote reasoning into a final content — see StreamEvent::ReasoningDelta for why streaming consumers of reasoning-only models (GLM-4.7) must accumulate both ReasoningDelta and Delta to reconstruct the answer.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§