Struct NativeProvider 

pub struct NativeProvider { /* private fields */ }

Provider adapter: an in-process NativeLlm behind the async LlmProvider trait. Generation is CPU-bound and &mut, so it runs under a mutex on a blocking thread.

Implementations

impl NativeProvider

pub fn new(llm: NativeLlm) -> NativeProvider

Trait Implementations

impl LlmProvider for NativeProvider

fn name(&self) -> &str

fn synthesizes(&self) -> bool

Whether this provider produces trustworthy natural-language synthesis of tool results. Large models (Bedrock, big OpenAI-compatible) do; small on-device tool-callers (Needle, the native 1.7B) do NOT — for those the agent renders a deterministic template from the program result instead of trusting the model to phrase the numbers. Default true.

fn chat<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, system: &'life1 str, msgs: &'life2 [Msg], tools: &'life3 [ToolSpec], ) -> Pin<Box<dyn Future<Output = Result<Turn, String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

One turn: given the system prompt, conversation, and available tools, return the model’s reply (assistant text and/or tool calls).

fn chat_json<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>( &'life0 self, system: &'life1 str, msgs: &'life2 [Msg], schema: &'life3 Value, name: &'life4 str, ) -> Pin<Box<dyn Future<Output = Result<Option<Value>, String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, 'life4: 'async_trait,

One turn whose output is constrained to a JSON schema by the decoder, rather than requested by a tool definition and hoped for.