inference-gateway-sdk 0.12.0

Rust SDK for interacting with various language models through the Inference Gateway
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with
code in this repository.

## Project Overview

Rust SDK for the [Inference Gateway][ig] - a unified interface for interacting
with various LLM providers (OpenAI, Anthropic, Groq, Ollama, Cohere,
Cloudflare, DeepSeek, Google).

[ig]: https://github.com/inference-gateway/inference-gateway

## Build & Development Commands

```bash
# Build
cargo build

# Run tests
cargo test --all-targets --all-features

# Run a single test
cargo test test_name

# Lint (format check)
cargo fmt --all -- --check

# Static analysis
cargo clippy --all-targets --all-features -- -D warnings

# Format code
cargo fmt
```

Using Taskfile (if `task` is installed):

```bash
task test      # Run tests
task lint      # Run linter
task analyse   # Run static analysis
```

## Architecture

This is a single-file library crate (`src/lib.rs`) with ~1900 lines containing:

### Core Types

- **`InferenceGatewayClient`**: HTTP client wrapping reqwest, configured with
  base URL and optional auth token
- **`InferenceGatewayAPI`**: Trait defining the API interface (list_models,
  generate_content, generate_content_stream, list_tools, health_check)
- **`Provider`**: Enum of supported LLM providers (Ollama, Groq, OpenAI,
  Cloudflare, Cohere, Anthropic, Deepseek, Google)
- **`GatewayError`**: Error enum using thiserror for HTTP status codes and
  request errors

### Request/Response Types

- `Message`, `MessageRole`: Chat message structures
- `CreateChatCompletionRequest/Response`: Non-streaming generation
- `CreateChatCompletionStreamResponse`, `SSEvents`: Streaming via SSE
- `Tool`, `FunctionObject`, `ToolType`: Function calling support
- `ListModelsResponse`, `ListToolsResponse`: API listing responses

### Key Patterns

- Builder pattern for client configuration: `.with_token()`, `.with_tools()`, `.with_max_tokens()`
- Async/await with tokio runtime
- Streaming uses `async_stream` crate with `futures_util::Stream`
- All serialization via serde with `#[serde(rename_all = "lowercase")]` for enums

## Testing

Tests use `mockito` for HTTP mocking. Tests are in `#[cfg(test)] mod tests`
at bottom of `lib.rs`. Run with:

```bash
cargo test --verbose
```

## Commit Message Convention

Use conventional commits: `feat:`, `fix:`, `refactor:`, `docs:`, `style:`,
`test:`, `chore:`, `ci:`, `perf:`