Module llm 

LLM provider bridge (OpenAI, Anthropic, local models). LLM Provider Bridge — FR-012

Integration layer for calling LLMs from within the GPU actor system. Each provider implements the LlmProvider trait. Requests and responses flow as K2K messages, making LLM calls a natural actor interaction.

Supported Providers

• OpenAI: GPT-4, GPT-3.5, embeddings
• Anthropic: Claude (messages API, tool use)
• Local: Ollama, vLLM, llama.cpp via HTTP

Architecture

GPU Actor → K2K "LLM Request" → LLM Bridge Actor → HTTP → Provider
                                                         ↓
GPU Actor ← K2K "LLM Response" ← LLM Bridge Actor ← HTTP ← Provider

Structs

ChatMessage

A chat message.

CompletionRequest

An LLM completion request.

CompletionResponse

An LLM completion response.

EchoProvider

Test LLM provider that echoes the input. Useful for testing the bridge.

EmbeddingRequest

An embedding request.

EmbeddingResponse

An embedding response.

LlmProviderMetrics

LLM provider metrics.

TokenUsage

Token usage tracking.

ToolCall

A tool call made by the LLM.

ToolDefinition

A tool/function definition for LLM function calling.

Enums

FinishReason

Why the LLM stopped generating.

LlmError

LLM errors.

Traits

LlmProvider

Trait for LLM provider implementations.