llmy 0.15.4

All-in-one LLM utilities.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134

//! High-level umbrella crate for building LLM-driven agents with tools, memory, and tokenization.
//!
//! This crate re-exports the main `llmy-*` crates behind a single top-level API so downstream
//! users can build an agent without having to depend on each sub-crate individually.
//!
//! # Building An Agent
//!
//! The smallest useful agent needs three pieces:
//!
//! 1. A system prompt.
//! 2. A [`agent::tool::ToolBox`] containing zero or more tools.
//! 3. A [`harness::Agent`] to hold conversation state and orchestrate tool calls.
//!
//! ```no_run
//! use llmy::agent::tool::ToolBox;
//! use llmy::agent::tools::files::ReadFileTool;
//! use llmy::harness::Agent;
//!
//! let mut tools = ToolBox::new();
//! tools.add_tool(ReadFileTool::new(std::env::current_dir().unwrap()));
//!
//! let agent = Agent::new(
//!     "You are a helpful assistant.".to_string(),
//!     tools,
//!     "docs-example".to_string(),
//! );
//!
//! let _ = agent;
//! ```
//!
//! Once the agent exists, you typically:
//!
//! 1. Create an [`client::client::LLM`] from CLI-style configuration in [`clap`] or directly from
//!    [`client`] primitives.
//! 2. Push user input with [`harness::Agent::step_with_user`].
//! 3. Continue stepping while the agent is still issuing tool calls.
//!
//! # Memory-Enabled Agents
//!
//! If you want the agent to search and update structured memory, construct an
//! [`agent::tools::memory::AgentMemoryContext`] and then build the agent with
//! [`harness::Agent::with_memory`].
//!
//! Requires the `memory-embed-search` cargo feature on `llmy` (it gates the
//! local embedding model — `ort-sys` does not support musl, hence the opt-in).
//!
//! ```no_run
//! # #[cfg(feature = "memory-embed-search")] {
//! use llmy::agent::tool::ToolBox;
//! use llmy::agent::tools::memory::{
//!     AgentMemory,
//!     AgentMemoryContext,
//!     embed::{SimilarityModel, SimilarityModelConfig},
//! };
//! use llmy::harness::{Agent, memory::AgentMemorySystemPromptCriteria};
//!
//! async fn build_agent() -> Result<Agent, llmy::LLMYError> {
//!     let memory = AgentMemoryContext::new(
//!         AgentMemory::default(),
//!         SimilarityModel::new(SimilarityModelConfig::default()).await?,
//!     );
//!
//!     Ok(Agent::with_memory(
//!         "You are a helpful assistant.".to_string(),
//!         ToolBox::new(),
//!         "docs-memory-example".to_string(),
//!         &memory,
//!         &AgentMemorySystemPromptCriteria::default(),
//!     )
//!     .await)
//! }
//! # }
//! ```
//!
//! # Module Guide
//!
//! - [`clap`] contains CLI-oriented configuration helpers that can build an LLM client from flags
//!   and environment variables.
//! - [`client`] contains the lower-level LLM client, billing, settings, debug, and model modules.
//! - [`agent`] contains the core tool traits and the aggregated tool modules used by agents.
//! - [`ebmed`] re-exports the embedding helpers used by memory search and similarity matching.
//! - [`harness`] contains the concrete in-memory agent implementation.
//! - [`tokenizer`] contains model metadata and token counting helpers.
//! - [`openai`] re-exports `async-openai` for callers that need direct access to request and
//!   response types.

/// Command-line and environment-driven LLM configuration helpers.
pub mod clap {
    pub use llmy_clap::*;
}

/// Lower-level client, model, billing, debug, and settings modules used to talk to LLM backends.
pub mod client {
    pub use llmy_client::*;
}

/// Core agent traits plus the bundled tool modules used by `llmy` agents.
pub mod agent {
    /// Tool implementations that can be installed into an [`super::agent::tool::ToolBox`].
    pub mod tools {
        pub use llmy_agent_tools::*;
    }
    pub use llmy_agent::*;
}

/// Embedding and similarity helpers used by memory search, token counting, and input truncation.
/// Only available when the `embed` (or `memory-embed-search`) feature is enabled.
#[cfg(feature = "embed")]
pub mod ebmed {
    pub use llmy_agent_tools::memory::embed::*;
}

/// Concrete in-memory agent harness with context management, compaction, and optional memory.
pub mod harness {
    pub use llmy_harness::*;
}

/// Tokenizer helpers and model metadata for approximate token counting and context sizing.
pub mod tokenizer {
    pub use llmy_tokenizer::*;
}

/// Common llmy error type shared across agent, tool, and client layers.
pub use llmy_types::error::LLMYError;

/// Raw `async-openai` re-export, kept only for transport plumbing (`Client`,
/// `OpenAIConfig`, `AzureConfig`, errors). All chat-completion request/response
/// types are now mirrored under [`client::req`] and [`client::resp`] with
/// `WithOtherFields` wrappers, so unknown JSON fields (e.g. provider extras
/// such as Gemini's `thought_signature`) are preserved on every round trip.
/// New code should prefer the mirrored types over anything from this module.
pub mod openai {
    pub use async_openai::*;
}