Struct RemoteLlmClient 

pub struct RemoteLlmClient { /* private fields */ }

LlmClient impl backed by a subprocess plugin’s stdio bridge.

Implementations

impl RemoteLlmClient

pub fn new( provider: String, model: String, plugin_id: String, stdin_tx: Sender<Value>, pending: Arc<DashMap<u64, Sender<Result<Value, String>>>>, streaming_pending: Arc<DashMap<u64, StreamingPending>>, next_id: Arc<AtomicU64>, ) -> Self

pub fn plugin_id(&self) -> &str

Trait Implementations

impl LlmClient for RemoteLlmClient

fn provider(&self) -> &str

Short identifier of the provider (“minimax”, “openai”, “stub”, …). Used as a Prometheus label; must be low-cardinality.

fn model_id(&self) -> &str

fn chat<'life0, 'async_trait>( &'life0 self, req: ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn stream<'a, 'async_trait>( &'a self, req: ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<BoxStream<'a, Result<StreamChunk>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'a: 'async_trait,

Incremental response stream. Default implementation delegates to chat() and emits a single TextDelta (or ToolCall* triples) followed by Usage + End. Providers that speak SSE override this to stream real token-level deltas.

fn embed<'life0, 'life1, 'async_trait>( &'life0 self, _texts: &'life1 [String], ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Compute dense vector embeddings for a batch of input texts. Default returns an error — providers that do not expose an embedding endpoint (or where it is not yet wired) opt out by leaving this default.