toast-api 0.1.7

# Toast - LLM CLI & API Server

An unofficial CLI client and API server for Claude and DeepSeek that provides both a command-line interface and OpenAI-compatible API endpoints.

## Overview

Toast is a versatile tool that provides multiple interfaces for working with LLMs:
1. A CLI client for direct interaction with Claude or DeepSeek (default)
2. An API server that provides OpenAI-compatible endpoints
3. Support for multiple LLM providers: Claude (Anthropic) and DeepSeek
4. Built-in coding agent capabilities with file operations and command execution

## Features

- Interactive CLI interface for both Claude and DeepSeek
- OpenAI-compatible API server
- Support for multiple models (Opus, Sonnet, Haiku variants)
- Built-in coding agent with tool use (exec commands, file reading)
- Conversation state management
- Model mapping (gpt-4/gpt-4.1 → Claude models)
- CORS support for browser clients
- SHA3 proof-of-work solver for DeepSeek's anti-bot protection

## Installation

```bash
# Clone the repository
git clone https://github.com/yourusername/toast.git
cd toast

# Build the project
cargo build --release
```

## Usage

### CLI Mode (Default)

By default, Toast runs in CLI mode with Claude. You can switch between providers:

```bash
# Use Claude (default)
./target/release/toast

# Use DeepSeek
./target/release/toast --deepseek
```

CLI options:

```bash
# Use Claude Opus model (claude-opus-4-20250514)
./target/release/toast --opus

# Use Claude Haiku model (claude-3-5-haiku-20241022)
./target/release/toast --haiku

# Use DeepSeek with R1 reasoning model (default for DeepSeek)
./target/release/toast --deepseek

# Use DeepSeek lite model
./target/release/toast --deepseek --haiku
```

#### CLI Commands

Toast includes a built-in coding agent that can execute commands and read files. These features work with both Claude and DeepSeek:

- Enter your message and press Enter to send to the LLM
- Use `/exit` or `exit` or `x` to quit
- The LLM can use `# exec <command>` to execute shell commands
- The LLM can use `# read_file <filename>` to read files
- Supports multiline commands using heredocs
- Automatic command execution based on LLM responses

### API Server Mode

Run Toast as an OpenAI-compatible API server:

```bash
# Start the API server
./target/release/toast --serve

# Custom port
./target/release/toast --serve --port 8080

# Custom model
./target/release/toast --serve --model claude-sonnet-4-20250514-claude-ai

# Custom host address
./target/release/toast --serve --host 127.0.0.1
```

### API Endpoint

The server exposes the following OpenAI-compatible endpoints:

- `POST /v1/chat/completions` - Chat completions API
- `POST /chat/completions` - Alternative endpoint
- `POST /` - Root endpoint (same as chat completions)
- `GET /health` - Health check endpoint

### API Reference

#### Chat Completions

```
POST /v1/chat/completions
```

Request body:

```json
{
  "model": "gpt-4.1",
  "messages": [
    {
      "role": "developer",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Hello!"
    }
  ]
}
```

Response:

```json
{
  "id": "f52a7702-6588-4b68-a057-bc2630d223b8",
  "object": "chat.completion",
  "created": 1683989000,
  "model": "claude-opus-4-20250514",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! I'm Claude, a helpful AI assistant. How can I assist you today?"
      }
    }
  ]
}
```

The API supports various content formats:
- Simple string content
- Array of strings
- Structured content objects
- Content blocks with type fields

### Model Mapping

The server maps OpenAI model names to Claude models:

- `gpt-3.5-turbo` → Claude Haiku (claude-3-5-haiku-20241022)
- `gpt-4`, `gpt-4-turbo` → Claude Sonnet (claude-sonnet-4-20250514-claude-ai)
- `gpt-4o`, `gpt-4.1` → Claude Opus (claude-opus-4-20250514)

You can also specify a Claude model directly in the model field.

### Authentication

The API server currently doesn't require authentication tokens from clients. However, you need to configure credentials for the underlying LLM providers (see Configuration section).

### Configuration

Toast requires authentication credentials for each LLM provider stored in the config directory:

#### Claude Configuration

- **macOS**: `~/Library/Application Support/toast/`
- **Linux**: `~/.config/toast/`
- **Windows**: `C:\Users\{USERNAME}\AppData\Roaming\toast\`

Required files:
- `cookie`: Contains your Claude session cookie
- `org_id`: Your Claude organization ID (automatically extracted from cookie if not present)

#### DeepSeek Configuration

- **macOS**: `~/Library/Application Support/toast/deepseek/`
- **Linux**: `~/.config/toast/deepseek/`
- **Windows**: `C:\Users\{USERNAME}\AppData\Roaming\toast\deepseek\`

Required files:
- `auth_token`: Your DeepSeek authentication token (from Authorization header)
- `cookies.json`: DeepSeek cookies including Cloudflare verification

To get your DeepSeek auth token:
1. Go to chat.deepseek.com in your browser
2. Open Developer Tools (F12)
3. Go to Network tab
4. Look for Authorization header in any request
5. Save the token part (without 'Bearer ') to the auth_token file

## Technical Details

- Built with Rust using Tokio for async runtime
- Uses Axum for the HTTP server
- Implements OpenAI's chat completion API specification
- Supports streaming responses in CLI mode
- Includes SHA3 proof-of-work solver for DeepSeek's anti-bot protection
- WebAssembly support for optimized cryptographic operations
- Conversation history tracking with context-based matching

## Development

### Building from source

```bash
cargo build
```

### Running tests

```bash
cargo test
```

## Dependencies

Key dependencies include:
- `rquest`: HTTP client with browser emulation
- `axum`: Web framework for the API server
- `tokio`: Async runtime
- `clap`: Command-line argument parsing
- `wasmtime`: WebAssembly runtime for DeepSeek POW
- `sha3`: Cryptographic hashing for DeepSeek verification
- `dashmap`: Concurrent hashmap for conversation tracking

## License

MIT