Skip to main content

Module llm

Module llm 

Source
Expand description

solo-api LLM clients that depend on the MCP transport.

Other LLM clients (Anthropic, OpenAI, Ollama) live in solo-storage::llm because they’re HTTP-only and storage-layer-callable. The MCP-sampling client lives here because it needs rmcp::Peer<RoleServer> — only available inside the MCP transport.

v0.9.0 P2 ships sampling::SamplingLlmClient + the sampling::SamplingClient trait the test fixture ([super::test_support::fake_mcp_client::FakeMcpClient]) implements.

Re-exports§

pub use sampling::DEFAULT_SAMPLING_TIMEOUT;
pub use sampling::SamplingClient;
pub use sampling::SamplingError;
pub use sampling::SamplingLlmClient;
pub use sampling::build_sampling_steward;
pub use sampling_coordinator::DEFAULT_COALESCE_MAX_BATCH;
pub use sampling_coordinator::DEFAULT_COALESCE_WINDOW;
pub use sampling_coordinator::SamplingCoordinator;

Modules§

sampling
SamplingLlmClientLlmClient impl backed by an MCP client’s sampling/createMessage capability.
sampling_coordinator
SamplingCoordinator — coalesces N concurrent SamplingClient::create_message calls into ⌈N/M⌉ calls within a configurable time window or batch-size limit.