Struct LLMRequest 

pub struct LLMRequest {

    pub messages: Arc<Vec<Message>>,
    pub system_prompt: Option<Arc<str>>,
    pub tools: Option<Arc<Vec<ToolDefinition>>>,
    pub model: String,
    pub max_tokens: Option<u32>,
    pub temperature: Option<f32>,
    pub stream: bool,
    pub output_format: Option<Value>,
    pub tool_choice: Option<ToolChoice>,
    pub parallel_tool_calls: Option<bool>,
    pub parallel_tool_config: Option<Box<ParallelToolConfig>>,
    pub reasoning_effort: Option<ReasoningEffortLevel>,
    pub effort: Option<String>,
    pub verbosity: Option<VerbosityLevel>,
    pub do_sample: Option<bool>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub presence_penalty: Option<f32>,
    pub frequency_penalty: Option<f32>,
    pub stop_sequences: Option<Vec<String>>,
    pub thinking_budget: Option<u32>,
    pub betas: Option<Vec<String>>,
    pub context_management: Option<Value>,
    pub coding_agent_settings: Option<Box<CodingAgentSettings>>,
    pub metadata: Option<Value>,
    pub previous_response_id: Option<String>,
    pub response_store: Option<bool>,
    pub responses_include: Option<Vec<String>>,
    pub service_tier: Option<String>,
    pub prompt_cache_key: Option<String>,
    pub prompt_cache_profile: Option<PromptCacheProfile>,
    pub fallbacks: Option<Vec<FallbackModel>>,
    pub fallback_credit_token: Option<String>,
    pub anthropic_request_overrides: Option<AnthropicRequestOverrides>,
}

Universal LLM request structure

Fields

messages: Arc<Vec<Message>>

Conversation history shared via Arc so per-turn request construction and continuation bookkeeping are O(1) instead of deep-copying the entire history (which grows unbounded over a session).

system_prompt: Option<Arc<str>>

Shared via Arc<str> (single allocation) so per-turn request construction can reuse the prompt envelope’s Arc<str> via a cheap refcount bump instead of cloning the full system prompt string.

tools: Option<Arc<Vec<ToolDefinition>>>

model: String

max_tokens: Option<u32>

temperature: Option<f32>

stream: bool

output_format: Option<Value>

Optional structured output JSON schema to request from providers that support it For Anthropic this will be sent as output_config.format = { type: "json_schema", schema: ... }

tool_choice: Option<ToolChoice>

Tool choice configuration based on official API docs Supports: “auto” (default), “none”, “any”, or specific tool selection

parallel_tool_calls: Option<bool>

Whether to enable parallel tool calls (OpenAI specific)

parallel_tool_config: Option<Box<ParallelToolConfig>>

Parallel tool use configuration following Anthropic best practices

reasoning_effort: Option<ReasoningEffortLevel>

Reasoning effort level for models that support it (none, minimal, low, medium, high, xhigh) Applies to: Claude, GPT-5 family, Gemini, Qwen3, DeepSeek with reasoning capability

effort: Option<String>

Effort level for overall token usage (low, medium, high, xhigh, max) Applies to: Anthropic adaptive-thinking models such as Claude Opus 4.7 Controls how many tokens Claude uses when responding, trading off between response thoroughness and token efficiency.

verbosity: Option<VerbosityLevel>

Verbosity level for output text (low, medium, high) Applies to: GPT-5.4-family Responses workflows and other models that support verbosity control

do_sample: Option<bool>

Advanced generation parameters

top_p: Option<f32>

top_k: Option<i32>

presence_penalty: Option<f32>

frequency_penalty: Option<f32>

stop_sequences: Option<Vec<String>>

thinking_budget: Option<u32>

Optional budget for extended thinking (Anthropic specific) Minimum value: 1024

betas: Option<Vec<String>>

Optional beta headers for Anthropic (and potentially others)

context_management: Option<Value>

Optional provider-specific context management configuration (Anthropic compaction/editing).

coding_agent_settings: Option<Box<CodingAgentSettings>>

Optional coding agent specific settings

metadata: Option<Value>

Optional turn metadata for git context (remote URLs, commit hash, etc.) This is sent as X-Turn-Metadata header to providers that support it

previous_response_id: Option<String>

Optional Responses API continuity pointer for server-side context chaining. Used by providers that support stateful response continuation.

response_store: Option<bool>

Optional Responses API storage flag. When set, providers that support store pass this through directly.

responses_include: Option<Vec<String>>

Optional Responses API include fields (e.g. reasoning encrypted content). Passed through only for providers/APIs that support include selectors.

service_tier: Option<String>

Optional native OpenAI service_tier request parameter. Passed through only for native OpenAI endpoints that support service tiers.

prompt_cache_key: Option<String>

Optional provider routing hint for prompt cache stickiness. OpenAI uses this value to improve routing locality for repeated prefixes.

prompt_cache_profile: Option<PromptCacheProfile>

Optional request-scoped prompt cache profile for provider-specific TTL overrides.

fallbacks: Option<Vec<FallbackModel>>

Optional explicit fallback models for Anthropic server-side fallback. Overrides provider.anthropic.fallbacks; sent with the server-side-fallback-2026-06-01 beta header. Subject to the same gates as the config list (first-party endpoint, a model that supports server-side fallbacks, no credit-token retry) and the same entry validation; an empty list sends nothing.

fallback_credit_token: Option<String>

Optional opaque credit token from a refused request’s stop_details.fallback_credit_token. Echoed on the retry to avoid paying the prompt-cache cost twice. Sent with the fallback-credit-2026-07-01 beta header. The refused request must have carried that header or server-side-fallback-2026-07-01, which grants the same stop_details fields.

anthropic_request_overrides: Option<AnthropicRequestOverrides>

Optional Anthropic-specific request overrides used when request semantics must not inherit VT Code’s provider defaults, such as the Anthropic compatibility server.

Trait Implementations

impl Clone for LLMRequest

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LLMRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LLMRequest

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for LLMRequest

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for LLMRequest

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.