camel-component-llm

LLM integration component for rust-camel.

Features

Chat (streaming + materialized) — Stream tokens per-event or collect into a single response body
Embeddings — Generate vector embeddings from text
Tool calling — Pass tool definitions via header; component emits tool-call intents (route owns dispatch)
Multi-turn conversations — Full conversation history with system, user, assistant, and tool roles
Response cache — Materialized-only, single-flight via dashmap + tokio::sync::watch
Cost observability — Config-driven pricing table; estimated cost emitted as header + tracing::info!
Retry — Manual loop honoring provider retry_after over exponential backoff (ADR-0021)
Concurrency control — Producer semaphore caps in-flight requests; permit released during backoff

Quick Start

Add to Camel.toml:

[llm]
[[llm.providers]]
name = "my-openai"
adapter = "openai"
model = "gpt-4o"
api_key = "${OPENAI_API_KEY}"

[[llm.providers]]
name = "local"
adapter = "ollama"
model = "llama3"
base_url = "http://localhost:11434/v1"

Or use LlmBundle programmatically:

use camel_component_llm::LlmBundle;
ctx.add_bundle(LlmBundle::from_toml(config))?;

URI Scheme

llm:{operation}?provider={name}&model={model}&temperature={n}&max_tokens={n}&stream={bool}&system_prompt={text}&timeout_secs={n}

Operations

Operation	Description
`llm:chat`	Chat completion. Defaults to streaming (`stream=true`)
`llm:embed`	Text embedding. Always materialized

URI Parameters

Parameter	Default	Description
`provider`	(required)	Provider name from config
`model`	provider default	Override model
`temperature`	provider default	Sampling temperature
`max_tokens`	provider default	Max output tokens
`stream`	`true`	`true` = streaming, `false` = materialized
`system_prompt`	—	System prompt override
`timeout_secs`	provider default	Activity timeout (streaming) or total deadline (materialized)

Exchange Headers

See src/headers.rs for the full list. Key headers:

Header	Direction	Description
`CamelLlmProvider`	Input/Output	Provider name
`CamelLlmModel`	Input/Output	Model name
`CamelLlmStream`	Input	`true` = streaming mode
`CamelLlmSystemPrompt`	Input	Override system prompt
`CamelLlmTools`	Input	Tool definitions (JSON array)
`CamelLlmToolChoice`	Input	Tool selection strategy
`CamelLlmToolCalls`	Output	Tool call intents emitted by the model
`CamelLlmTokensIn`	Output	Input token count (materialized only)
`CamelLlmTokensOut`	Output	Output token count (materialized only)
`CamelLlmFinishReason`	Output	Finish reason (`stop`, `length`, `tool_calls`)
`CamelLlmUsageAvailable`	Output	Whether usage/headers are populated
`CamelLlmEstimatedCostUsd`	Output	Estimated cost in USD (materialized only)
`CamelLlmMessages`	Input	Multi-turn conversation history
`CamelLlmText`	Output	Response text

Provider Feature Flags

Provider	Cargo feature	Requires siumai
Mock	`mock` (default)	No
OpenAI	`openai`	Yes
Ollama	`ollama`	Yes

Enable multiple: --features "openai,ollama" or use all-providers.

Architecture Decisions

ADR-0020 — Project-owned LlmProvider trait isolates siumai SDK behind two files
ADR-0021 — Manual retry loop honors retry_after over exponential backoff

Example

See examples/llm-example/ for a working chat example.

License

Licensed under either of Apache License, Version 2.0 or MIT License at your option.

camel-component-llm 0.18.0