model-gateway-rs 0.1.6

A Rust library for model gateway services, providing traits and SDKs for various AI models.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

# model-gateway-rs

**model-gateway-rs** is a Rust library designed as a unified gateway for interacting with various machine learning model backends. It provides a clean abstraction over different model clients (e.g., OpenAI, LLaMA, Gemini) so you can integrate multiple models through a single consistent interface.

## Features
- Unified trait-based API for model inference
- Supports text generation, embeddings, vision models (extensible)
- Easy to plug in new model clients (OpenAI, custom LLMs, etc.)
- Async-friendly, built with `async-trait`
- Suitable for integration into larger systems (e.g., agents, API servers)

## Directory structure
```
src/
├── clients/      # Concrete client implementations (e.g., OpenAI)
├── sdk/          # High-level SDK for external usage
├── traits/       # Core model traits (text, embed, vision)
├── types/        # Shared types (input/output data structures)
└── lib.rs        # Library entry point
```

## Usage
Add to your `Cargo.toml`:
```toml
model-gateway = { git = "https://github.com/code-serenade/model-gateway-rs" }
```

Example:
```rust
use model_gateway::traits::text::TextGeneration;

async fn run_inference(client: impl TextGeneration) {
    let prompt = TextPrompt {
        prompt: "Hello, world!".to_string(),
        system_prompt: None,
        temperature: None,
        top_p: None,
    };
    let result = client.infer_text(prompt).await.unwrap();
    println!("Result: {}", result.content);
}
```

## License
MIT