api_ollama 0.2.0

Ollama local LLM runtime API client for HTTP communication.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117

# Ollama Examples

## Purpose

Usage demonstrations for the Ollama API client showing real-world usage patterns and best practices.

## Organization Principles

- Examples demonstrate actual usage, not testing
- Each example focuses on a specific use case
- Examples are runnable with `cargo run --example <name>`
- All examples use real Ollama API (no mocks)

## Prerequisites

**IMPORTANT:** Ollama is a local LLM runtime that requires installation and setup before examples will work.

### Step 1: Install Ollama

Download and install from https://ollama.ai

- **macOS**: `brew install ollama`
- **Linux**: `curl -fsSL https://ollama.ai/install.sh | sh`
- **Windows**: Download installer from https://ollama.ai/download

### Step 2: Start Ollama Server

```bash
# Start Ollama server (runs in background)
ollama serve
```

Server runs on `http://localhost:11434` by default.

### Step 3: Pull a Model

```bash
# Recommended model for examples
ollama pull llama3.2

# For vision examples
ollama pull llama3.2-vision

# For code examples
ollama pull codellama

# Verify installation
ollama list
```

### Step 4: Test Setup

```bash
# Quick test - should respond with a greeting
ollama run llama3.2 "Hello!"
```

## Running Examples

Once Ollama is installed, running, and has models downloaded:

```bash
# Basic examples
cargo run --example ollama_chat_basic --features full
cargo run --example ollama_chat_streaming --features full

# Interactive examples
cargo run --example ollama_chat_interactive --features full

# Advanced examples
cargo run --example ollama_code_assistant --features full
cargo run --example ollama_document_analyzer --features full
```

## Troubleshooting

### Examples timeout or "connection refused"

**Cause:** Ollama server not running

**Fix:**
```bash
# Check if server is running
curl http://localhost:11434/api/tags

# Start server if not running
ollama serve
```

### "Model not found" error

**Cause:** Model not downloaded

**Fix:**
```bash
ollama pull llama3.2
```

### Examples hang on first run

**Cause:** Ollama loading model into memory (first-time setup is slow)

**Solution:** Wait 30-60 seconds. Subsequent runs will be faster.

## Requirements

- Ollama server running (default: http://localhost:11434)
- At least one model pulled (e.g., `ollama pull llama3.2`)
- Set `OLLAMA_HOST` environment variable if using non-default endpoint
- Minimum 8GB RAM (16GB+ recommended for larger models)

## Key Differences from Cloud APIs

- **No API Key:** Runs locally, no authentication needed
- **Offline Capable:** Works without internet after models are downloaded
- **Privacy:** All processing happens on your machine
- **Performance:** Depends on your hardware (CPU/GPU/RAM)