ambi 0.1.5

A flexible, multi-backend, customizable AI agent framework, entirely based on Rust.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112

# Ambi πŸ¦€


> [δΈ­ζ–‡](README_zh.md)

Ambi is a flexible, highly customizable AI Agent framework built entirely in Rust. It features a dual-engine
architecture supporting both local inference and cloud APIs, alongside a powerful multi-tool calling system and
intelligent context management.

## ✨ Key Features


- **Dual-Engine Architecture**:
    - **Local Engine**: Powered by `llama.cpp`, supporting GGUF models with hardware acceleration for CUDA, Vulkan,
      Metal, etc.
    - **Cloud Engine**: Fully compatible with OpenAI-spec APIs (e.g., DeepSeek, SiliconFlow, Groq).
- **Parallel Multi-Tool Calling**: Capable of parsing and executing multiple `[TOOL_CALL]` instructions in a single
  response, significantly boosting complex task efficiency.
- **Fine-grained Tool Control**: Configure `timeout_secs` and `max_retries` independently for each tool.
- **Built-in Chat Templates**: Native support for Chatml, Llama3, Gemma, DeepSeek, and more, with support for fully
  custom templates.
- **Intelligent Context Eviction**: A safe eviction algorithm that identifies `User` messages as cutting points to
  prevent token overflow while maintaining logical flow.
- **Streaming Reasoning Support**: Automatically identifies and formats reasoning tags like `<think>` for models like
  DeepSeek.

## πŸ“¦ Installation


Add Ambi to your `Cargo.toml`:

```toml
[dependencies]
ambi = "0.1.3"

# For cloud-only usage (faster compilation):

# ambi = { version = "0.1.3", default-features = false, features = ["openai-api"] }

```

## πŸš€ Quick Start


### 1. Instantiate an Agent


```rust
use ambi::{Agent, EngineConfig, OpenAIEngineConfig};
use ambi::llm::chat_template::ChatTemplateType;

#[tokio::main]

async fn main() -> anyhow::Result<()> {
    let config = EngineConfig::OpenAI(OpenAIEngineConfig {
        api_key: "your-api-key".into(),
        base_url: "[https://api.deepseek.com](https://api.deepseek.com)".into(),
        model_name: "deepseek-chat".into(),
        temp: 0.7,
        top_p: 0.9,
    });

    let mut agent = Agent::make(config)?
        .template(ChatTemplateType::Deepseek)
        .preamble("You are a helpful assistant.");

    let response = agent.chat("Hello!").await;
    println!("{}", response);

    Ok(())
}
```

### 2. Define a Custom Tool


```rust
use ambi::agent::tool::{Tool, ToolDefinition, ToolErr};
use async_trait::async_trait;
use serde::Deserialize;

#[derive(Deserialize)]

struct AddArgs {
    a: f64,
    b: f64
}

struct AddTool;

#[async_trait]

impl Tool for AddTool {
    const NAME: &'static str = "add";
    type Args = AddArgs;
    type Output = f64;

    fn definition(&self) -> ToolDefinition {
        ToolDefinition::new(Self::NAME, "Add two numbers together")
            .parameter("a", "number", "First number")
            .parameter("b", "number", "Second number")
    }

    async fn call(&self, args: Self::Args) -> Result<Self::Output, ToolErr> {
        Ok(args.a + args.b)
    }
}

// Mount when creating Agent
// let mut agent = Agent::make(config)?.tool(AddTool)?;
```

## πŸ› οΈ Hardware Acceleration (Local)


Enable specific features for GPU acceleration:

- **CUDA**: `cargo build --features cuda`
- **Metal**: `cargo build --features metal`
- **Vulkan**: `cargo build --features vulkan`

## βš–οΈ License


Licensed under the **Apache-2.0** License.