kova

Async-first Rust library for building LLM-powered agents. Trait-based architecture with pluggable providers, tool calling, structured output, prompt caching, embeddings, pull-based streaming, thinking-model support, MCP integration, and telemetry. The agent is stateless — the host owns conversation history.

Installation

[dependencies]
kova-sdk = "0.1"

# With OpenTelemetry tracing
kova-sdk = { version = "0.1", features = ["telemetry"] }

Or with cargo:

cargo add kova-sdk

Architecture

kova
├── agent        # Agent + AgentBuilder — the main orchestration loop
├── provider     # LlmProvider trait + OpenAI / Bedrock / Gemini / Ollama implementations
├── tool         # Tool trait + thread-safe ToolRegistry
├── tools        # Built-in fs/shell/web tools (feature `tools` / `web-tools`)
├── mcp          # MCP client (stdio / HTTP+SSE / Streamable HTTP + OAuth) + McpTool adapter
├── streaming    # SSE / line-framing helpers for provider streams
├── telemetry    # TelemetryConfig + MetricsCollector
├── models       # Shared data types (messages, content blocks, events)
└── error        # Unified KovaError enum

Providers

Chain-of-thought output from thinking models is returned in ModelResponse::thinking and never stored in conversation history. During streaming it arrives as AgentEvent::ThinkingDelta.

Quick Start

use std::sync::Arc;
use kova_sdk::prelude::*;
use kova_sdk::provider::openai::{OpenAiCompatibleProvider, OpenAiProviderConfig};

#[tokio::main]
async fn main() -> Result<(), KovaError> {
    let config = OpenAiProviderConfig::new("http://127.0.0.1:1234", "my-model");
    let provider = Arc::new(OpenAiCompatibleProvider::new(config)?);

    let agent = AgentBuilder::new().provider(provider).build()?;
    let messages = vec![user_message("Hello!")];
    let response = agent.run(&messages).await?;
    println!("{}", response.text);
    Ok(())
}

Stateless by design

Agent::run is the core primitive: you own the conversation history, the agent owns the turn. Nothing is read from or written to any store — persistence, sessions, and compaction belong to the host.

let mut history = vec![user_message("What's the weather in Tokyo?")];
let response = agent.run(&history).await?;          // tools execute, loop runs
history.extend(response.new_messages);              // you persist
println!("{} ({} tokens)", response.text, response.usage.total_tokens);

Streaming

Pull-based, no handler registration needed:

use futures::StreamExt;
use kova_sdk::agent::AgentEvent;

let stream = agent.run_stream(&history);
futures::pin_mut!(stream);
while let Some(event) = stream.next().await {
    match event? {
        AgentEvent::TextDelta { text } => print!("{text}"),
        AgentEvent::ToolCallStarted { name, .. } => eprintln!("⚙ {name}…"),
        AgentEvent::TurnCompleted { response } => history.extend(response.new_messages),
        _ => {}
    }
}

Reliability

Transient provider failures (connection errors, timeouts, 408/429/5xx) are retried automatically with exponential backoff — default 2 retries, configurable or disableable via AgentBuilder::retry_config(RetryConfig { .. }).

Feature Flags

Skip the AWS dependency tree entirely if you don't need Bedrock:

[dependencies]
kova-sdk = { version = "0.3", default-features = false, features = ["openai"] }

Without the telemetry feature, TelemetryConfig::init() installs a lightweight tracing_subscriber — zero OTEL overhead.

The built-in tools are opt-in to keep the core dependency-light. Enable them and register against an injected ToolPolicy that confines filesystem access and guards the web tools against SSRF:

[dependencies]
kova-sdk = { version = "0.3", features = ["web-tools"] }

use std::sync::Arc;
use kova_sdk::tools::{ToolPolicy, register_all_tools_with_policy};

let policy = Arc::new(ToolPolicy {
    workspace_root: Some("/srv/project".into()),
    ..ToolPolicy::default()
});

let mut builder = AgentBuilder::new().provider(provider);
for tool in register_all_tools_with_policy(policy) {
    builder = builder.tool(tool);
}
let agent = builder.build()?;

Documentation