Module provider 

Universal LLM provider abstraction with API-specific role handling

This module provides a unified interface for different LLM providers (OpenAI, Anthropic, Gemini) while properly handling their specific requirements for message roles and tool calling.

Message Role Mapping

Different LLM providers have varying support for message roles, especially for tool calling:

OpenAI API

• Full Support: system, user, assistant, tool
• Tool Messages: Must include tool_call_id to reference the original tool call
• Tool Calls: Only assistant messages can contain tool_calls

Anthropic API

• Standard Roles: user, assistant
• System Messages: Can be hoisted to system parameter or treated as user messages
• Tool Responses: Converted to user messages (no separate tool role)
• Tool Choice: Supports auto, any, tool, none modes

Gemini API

• Conversation Roles: Only user and model (not assistant)
• System Messages: Handled separately as systemInstruction parameter
• Tool Responses: Converted to user messages with functionResponse format
• Function Calls: Uses functionCall in model messages

Best Practices

1. Always use MessageRole::tool_response() constructor for tool responses
2. Validate messages using validate_for_provider() before sending
3. Use appropriate role mapping methods for each provider
4. Handle provider-specific constraints (e.g., Gemini’s system instruction requirement)

Example Usage

use vtcode_llm::provider::{Message, MessageRole};

// Create a proper tool response message
let tool_response = Message::tool_response(
    "call_123".to_string(),
    "Tool execution completed successfully".to_string()
);

// Validate for specific provider
tool_response.validate_for_provider("openai").unwrap();

Structs

AnthropicRequestOverrides

ChoiceDecisionAnswer

A validated choice answer. Confidence is retained without a threshold.

ChoiceDecisionRequest

Evidence and instructions for exactly one named choice question. Deliberately omits images, predicates, scores, and generation controls.

ChoiceDecisionResponse

An inconclusive answer still retains independently parsed billed usage.

CodingAgentSettings

Settings to refine model behavior for coding agent tasks

ContextWindowProvider

Provider-local context capacity discovered for the selected model. All transport and capability behavior remains delegated to the underlying provider.

DecisionChoiceOption

One supplied text category and its meaning.

DecisionProbability

A probability for one supplied category.

FallbackModel

Fallback model configuration for Anthropic server-side fallback. The explicit-list form of fallbacks, sent with the server-side-fallback-2026-06-01 beta header (the "default" keyword form uses server-side-fallback-2026-07-01; pairing a header with the other form is rejected).

FunctionCall

Function call within a tool call

FunctionDefinition

Function definition within a tool

GrammarDefinition

Grammar definition for GPT-5 context-free grammar (CFG) constraints

LLMErrorMetadata

LLMRequest

Universal LLM request structure

LLMResponse

Universal LLM response structure

Message

Universal message structure supporting both text and image content

ParallelToolConfig

Configuration for parallel tool use behavior Based on Anthropic’s parallel tool use guidelines

PreparedResponsesRequest

ProviderCapabilities

Cached provider capabilities to reduce repeated trait method calls

ResponsesCompactionOptions

Optional overrides for standalone Responses compaction requests.

ResponsesContinuationState

SamplingOverrides

Per-model sampling parameter overrides resolved from a custom provider profile. Every field defaults to None, meaning the agent loop’s global configuration value applies unchanged.

ShellToolDefinition

Shell tool definition for GPT-5.1 shell tool type Allows controlled command-line interface interactions

SpecificFunctionChoice

Specific function choice details

SpecificToolChoice

Specific tool choice for forcing a particular function call

ToolCall

Universal tool call that matches OpenAI/Anthropic/Gemini specifications

ToolDefinition

Universal tool definition that matches OpenAI/Anthropic/Gemini specifications Based on official API documentation from Context7

ToolNamespace

A namespace grouping for deferred tools. When a tool has a namespace and defer_loading is true, only the namespace metadata (name + description) is sent upfront. The full tool definitions are loaded when the model searches for a tool in that namespace.

Usage

Enums

AnthropicOptionalStringOverride

AnthropicOptionalU32Override

AnthropicThinkingConfig

Anthropic thinking configuration for fallback models.

AnthropicThinkingDisplayOverride

AnthropicThinkingModeOverride

AssistantPhase

Phase metadata for assistant messages in multi-step Responses-style workflows.

ContentPart

Content type for messages that can include both text and images

FinishReason

ImageDetail

Detail level for image processing (DeepSeek/OpenAI detail field).

LLMError

LLM error types with optional provider metadata

LLMStreamEvent

MessageClearAt

Controls when an Anthropic mid-conversation system message is cleared.

MessageContent

MessageRole

NormalizedStreamEvent

PromptCacheProfile

ReasoningSource

Provider-side classification for streamed reasoning text.

ToolChoice

Tool choice configuration that works across different providers Based on OpenAI, Anthropic, and Gemini API specifications Follows Anthropic’s tool use best practices for optimal performance

ToolSearchAlgorithm

Tool search algorithm for Anthropic’s advanced-tool-use beta

Constants

DECISIONS_MODEL

The model supported by the experimental Decisions endpoint.

Traits

LLMProvider

Universal LLM provider trait

Functions

get_cached_capabilities

Extract and cache provider capabilities for a given provider and model

prepare_openai_responses_request

prepare_responses_continuation_request

records_responses_continuation_state

responses_continuation_key

supports_responses_chaining

uses_incremental_responses_history

Compatibility shim for older callers.

Type Aliases

BorrowedLLMStream

LLMNormalizedStream

LLMStream