# Design
## Architecture Overview
Kova is structured as a set of orthogonal, trait-based modules. Each module owns one concern; the `Agent` composes them at runtime.
```
┌──────────────────────────────────────────────────────────┐
│ Agent │
│ ┌──────────┐ ┌──────────────┐ ┌───────────────────┐ │
│ │ Provider │ │ ToolRegistry │ │ MemoryStore │ │
│ │(LlmProv.)│ │(RwLock<Map>) │ │ (RwLock<Map>) │ │
│ └──────────┘ └──────────────┘ └───────────────────┘ │
│ ┌──────────────────┐ ┌──────────────────────────────┐ │
│ │ StreamingHandler │ │ InferenceConfig │ │
│ └──────────────────┘ └──────────────────────────────┘ │
└──────────────────────────────────────────────────────────┘
│ composes
┌────────▼──────────────────────────────────────────────────┐
│ Orchestrator │
│ HashMap<String, Arc<Agent>> + timeout + OrchestratorPattern
└───────────────────────────────────────────────────────────┘
```
## Agentic Loop
The loop in `agent/mod.rs` is the core of the library:
```
chat(conversation_id, user_message)
│
├─ store user message in MemoryStore
│
└─ loop (up to max_iterations + 1 total provider calls):
├─ build messages = system_prompt + memory.get_history()
├─ provider.chat_completion(messages, tool_defs, config)
│
├─ StopReason::ToolUse?
│ ├─ store assistant message (tool-use blocks)
│ ├─ execute tools concurrently (Semaphore, default cap 10)
│ ├─ store each ToolResult as Role::Tool message
│ └─ continue loop
│
├─ StopReason::EndTurn | MaxTokens | Unknown?
│ ├─ store input_tokens in last_turn_input_tokens (AtomicU32) if provider reports usage
│ └─ store assistant message + return text
│
└─ loop limit hit → Err(KovaError::MaxIterations)
```
`chat_stream` is identical but uses `provider.chat_completion_stream` and forwards `StreamEvent`s to the registered `StreamingHandler` before returning the full text.
## Provider Abstraction
`LlmProvider` is an object-safe async trait. Both implementations (`OpenAiCompatibleProvider`, `BedrockProvider`) are stateless HTTP clients — all mutable state lives in the caller.
```
LlmProvider (trait, object-safe)
├── OpenAiCompatibleProvider ← reqwest + SSE parsing
├── BedrockProvider ← aws-sdk + SigV4 + ConverseStream
├── GeminiProvider ← reqwest + SSE (alt=sse), x-goog-api-key, thinking-model filtering
└── OllamaProvider ← reqwest + NDJSON streaming, no auth, /api/chat + /api/tags
```
**Streaming contract**: `chat_completion_stream` returns a `Pin<Box<dyn Stream<…> + Send>>`. The agent polls this stream, accumulates `ToolUseDelta` events into complete tool-call records, then executes them after the stream closes. `ThinkingDelta` events are forwarded to the `StreamingHandler` but otherwise ignored by the accumulator.
**OpenAI path override**: `OpenAiProviderConfig` exposes `with_chat_completions_path` and `with_models_path` so Azure deployments, local servers, and proxies can override the default `/v1/chat/completions` and `/v1/models` paths without subclassing. `with_reasoning_effort` sets the `reasoning_effort` field for o-series models.
**Bedrock credential resolution**: explicit credentials → named profile → SDK default chain. The `BedrockProvider` resolves credentials once at construction; re-construction is required to rotate them. `BedrockProviderConfig::with_additional_model_request_fields` passes arbitrary JSON as `additionalModelRequestFields` in the Converse API request — used for model-specific knobs such as `budgetTokens` for extended thinking on Claude models.
**Gemini path override**: `GeminiProviderConfig` exposes `with_base_url` and `with_api_version` for test servers and future API versions. Streaming requests append `?alt=sse` so the shared SSE parser handles Gemini stream chunks without a separate code path. `with_thinking_budget` controls the extended-thinking token budget (`-1` = dynamic, `0` = off, positive = cap).
**Ollama streaming**: uses newline-delimited JSON (NDJSON) over `/api/chat` rather than SSE. Each line is a complete `OllamaResponse` object; the final chunk has `done: true` and carries token counts. The `OllamaThink` enum serialises as a bool (`true`) or string (`"high"` / `"medium"` / `"low"`) to match Ollama's `think` field.
**Thinking/reasoning across all providers**: chain-of-thought output (OpenAI `reasoning_content`, Bedrock `ReasoningContent`, Gemini `thought: true`, Ollama `thinking` field) is extracted by each provider's converter and placed in `ModelResponse::thinking`. It is never written into the conversation history and is never re-submitted to the LLM. During streaming, reasoning text arrives as `StreamEvent::ThinkingDelta`. In the non-streaming `chat` path the agent replays any thinking text as `ThinkingDelta` + empty `ContentDelta` events to the registered `StreamingHandler` so callers see reasoning output in both modes.
## Tool Execution Model
```
ToolRegistry: Arc<RwLock<HashMap<String, Arc<dyn Tool>>>>
```
Clone of a `ToolRegistry` shares the same inner `Arc` — all clones see the same registrations. This allows the agent to pass a clone to concurrent tasks without re-locking the outer `Arc`.
Concurrent execution uses `tokio::sync::Semaphore` to cap parallelism. Tool results are collected with `futures::future::join_all` and stored in order.
## InferenceConfig
`InferenceConfig { model, max_tokens, temperature }` controls LLM call parameters. It is stored on `Agent` as a field (set via `AgentBuilder::inference_config(cfg)`) and cloned into each provider call inside the agentic loop. This replaces the earlier pattern of constructing `InferenceConfig::default()` inline on every iteration.
`AgentBuilder` defaults to `InferenceConfig::default()` (all fields `None`), so embedders that do not call `inference_config()` see no behaviour change.
## Memory Design
`MemoryStore` is keyed by `conversation_id`, allowing a single store instance to serve many concurrent conversations.
`InMemoryStore::with_max_messages(n)` truncates the oldest non-system messages when the count exceeds `n`. System prompt messages (Role::System) at position 0 are preserved across truncation to avoid breaking provider context requirements.
## MCP Integration
MCP tools are discovered at build time (via `AgentBuilder::mcp_client`) and registered as `McpTool` adapters in the `ToolRegistry`. At runtime they are indistinguishable from native tools — the agent calls `execute` and `McpTool` serialises the JSON-RPC request to the MCP server.
`McpClient` serialises all JSON-RPC calls through `Arc<Mutex<McpConnection>>`, since the MCP spec requires ordered request/response pairs over a single connection.
## Orchestrator Design
The `Orchestrator` holds a `HashMap<String, Arc<Agent>>`. Patterns reference agents by name (string keys). This decouples pattern configuration from agent construction and makes it easy to swap agents without rebuilding the orchestrator.
- **Sequential**: implemented as an iterative fold — no recursion, bounded stack.
- **Parallel**: uses `tokio::time::timeout` + `futures::future::join_all`; individual agent failures are captured rather than short-circuiting.
- **Router**: the router agent is called first; its response text is used as the key to look up the downstream agent name.
## Error Design
`KovaError` is a flat enum with `thiserror 2` derives. Each variant has a clear owner:
| Variant | Owner |
|---------|-------|
| `Provider`, `Connection`, `Http`, `Timeout` | Provider layer |
| `ToolExecution`, `ToolNotFound` | Tool layer |
| `Memory` | Memory layer |
| `Mcp` | MCP layer |
| `Stream` | Streaming layer |
| `Orchestration`, `MaxIterations` | Orchestrator / Agent loop |
| `Build` | AgentBuilder validation |
| `Serialization`, `Io` | Cross-cutting I/O |
`AgentBuilder::build()` is the only place where configuration errors (`Build` variant) can originate — everything else is a runtime error.
## Concurrency & Thread Safety
All public-facing types implement `Send + Sync`. The compile-time assertions in `tests/send_sync_assertions.rs` act as a regression guard — if a non-`Send` type is accidentally introduced, the build fails.
Internal synchronisation primitives:
| Type | Primitive | Reason |
|------|-----------|--------|
| `ToolRegistry` | `RwLock` | Many concurrent reads, infrequent writes |
| `InMemoryStore` | `RwLock` | Many reads, sequential append |
| `McpClient` | `Mutex` | Ordered JSON-RPC request/response |
| Tool parallelism | `Semaphore` | Bounded fan-out |
| `last_turn_input_tokens` | `AtomicU32` | Lock-free token counter read from any thread |
## Telemetry Design
The `telemetry` cargo feature gates all OTEL crates. This is a deliberate trade-off: OTEL crates are heavy (compile time + binary size); most embedders don't need them.
`TelemetryConfig::init()` installs either a full OTEL pipeline or a lightweight `tracing_subscriber` depending on the feature flag. The API surface is identical in both cases.
`MetricsCollector` uses atomic integers and a `RwLock<Vec<f64>>` for histograms. It intentionally does not integrate with OTEL metrics — it is a lightweight, always-available introspection tool, not a production metrics pipeline.