topkio 0.1.1

topkio is an innovative and high-performance large language model framework.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# topkio

an innovative LLM framework.  
Minimal footprint, high throughput, ease of use, and streaming capabilities.


## 📦 Installation

```bash
[dependencies]
topkio = "0.1.1"
```

## Quick Start
```rust
use topkio::OpenAIClient;

#[tokio::main]
async fn main() {
    let client = OpenAIClient::with_ollama("http://localhost:11434/v1");

    let agent_builder = client.config("llama3.2");
    let builder = agent_builder.stream(true);
    let builder = builder.temperature(0.8);
    let config = builder.build();

    let prompt = "1 + 1 = ";

    let _ = client
        .prompt(config, prompt, &mut |res| {
            print!("{}", res);
            Ok(())
        })
        .await;

    println!("\n");
}
```