#[non_exhaustive]pub struct GenerationOptions {
pub reasoning: Option<Reasoning>,
pub cache: Option<CacheRetention>,
pub service_tier: Option<ServiceTier>,
pub verbosity: Option<Verbosity>,
pub parallel_tool_calls: Option<bool>,
pub top_p: Option<f64>,
pub seed: Option<u64>,
pub stop: Vec<String>,
pub cache_key: Option<String>,
pub on_unsupported: Option<OnUnsupported>,
}Expand description
Provider-neutral generation knobs for one request. An unset field leaves
the provider’s default. A field the wire or model cannot honour is
reported through Self::unsupported_policy, never silently dropped.
This is the reusable value: build it once and hand it to an agent
(AgentBuilder::options), overlay a run’s on an agent’s
(Self::overlay), or store it serialized with a request. For a single
request, CompletionRequest, AgentBuilder and AgentRunner also have
a shortcut per field (.seed(7)) that writes that one field into their
options.
Fields (Non-exhaustive)§
This struct is marked as non-exhaustive
Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.reasoning: Option<Reasoning>How much the model reasons before answering.
cache: Option<CacheRetention>How long the provider keeps the prompt prefix cached.
service_tier: Option<ServiceTier>The processing tier the request asks for.
verbosity: Option<Verbosity>How long the answer should be.
parallel_tool_calls: Option<bool>Whether the model may call several tools in one turn.
top_p: Option<f64>Nucleus sampling probability mass.
seed: Option<u64>Sampling seed, for providers that offer best-effort determinism.
stop: Vec<String>Sequences that end generation. Empty means none.
cache_key: Option<String>A key the provider routes prompt-cache lookups by, such as a conversation’s id: requests with the same key reach the server that holds their prefix. A hint: providers that route without one send nothing for it.
on_unsupported: Option<OnUnsupported>What happens to an option the wire or model cannot honour. None
leaves the policy unset, which acts as OnUnsupported::Error
(Self::unsupported_policy); a set policy counts as setting an
option.
Implementations§
Source§impl GenerationOptions
impl GenerationOptions
Sourcepub fn new() -> GenerationOptions
pub fn new() -> GenerationOptions
No option set and no policy set; the same as Self::default.
Sourcepub fn is_default(&self) -> bool
pub fn is_default(&self) -> bool
Whether every field holds its default: no option set, and no policy set. A request whose options are default is not checked against its model’s catalog rules; setting any field, the policy included, turns the checks on.
Sourcepub fn unsupported_policy(&self) -> OnUnsupported
pub fn unsupported_policy(&self) -> OnUnsupported
The policy in effect: the one set, or OnUnsupported::Error.
Sourcepub fn reasoning(self, reasoning: impl Into<Reasoning>) -> GenerationOptions
pub fn reasoning(self, reasoning: impl Into<Reasoning>) -> GenerationOptions
Set the reasoning level or budget.
Sourcepub fn cache(self, cache: CacheRetention) -> GenerationOptions
pub fn cache(self, cache: CacheRetention) -> GenerationOptions
Set the cache retention.
Sourcepub fn service_tier(self, tier: ServiceTier) -> GenerationOptions
pub fn service_tier(self, tier: ServiceTier) -> GenerationOptions
Set the service tier.
Sourcepub fn verbosity(self, verbosity: Verbosity) -> GenerationOptions
pub fn verbosity(self, verbosity: Verbosity) -> GenerationOptions
Set the answer verbosity.
Sourcepub fn parallel_tool_calls(self, parallel: bool) -> GenerationOptions
pub fn parallel_tool_calls(self, parallel: bool) -> GenerationOptions
Allow or forbid several tool calls in one turn.
Sourcepub fn top_p(self, top_p: f64) -> GenerationOptions
pub fn top_p(self, top_p: f64) -> GenerationOptions
Set the nucleus sampling probability mass.
Sourcepub fn seed(self, seed: u64) -> GenerationOptions
pub fn seed(self, seed: u64) -> GenerationOptions
Set the sampling seed.
Sourcepub fn stop<S>(self, stop: impl IntoIterator<Item = S>) -> GenerationOptions
pub fn stop<S>(self, stop: impl IntoIterator<Item = S>) -> GenerationOptions
Replace the stop sequences.
Sourcepub fn cache_key(self, key: impl Into<String>) -> GenerationOptions
pub fn cache_key(self, key: impl Into<String>) -> GenerationOptions
Set the prompt-cache routing key.
Sourcepub fn on_unsupported(self, policy: OnUnsupported) -> GenerationOptions
pub fn on_unsupported(self, policy: OnUnsupported) -> GenerationOptions
Set what happens to an option the wire or model cannot honour.
Sourcepub fn fields(&self) -> OptionFields<'_>
pub fn fields(&self) -> OptionFields<'_>
Every option but the policy, borrowed, as a wire maps it.
Destructures self with no .., so a new field fails to compile
here first.
Sourcepub fn overlay(self, over: &GenerationOptions) -> GenerationOptions
pub fn overlay(self, over: &GenerationOptions) -> GenerationOptions
self with every field over sets put on top: a Some option, a
non-empty stop list, a set on_unsupported (so a run can restore
Error over an agent’s Ignore). Every other
field keeps self’s value. An agent’s options overlaid with a run’s
give the run’s where it sets one.
Trait Implementations§
Source§impl Clone for GenerationOptions
impl Clone for GenerationOptions
Source§fn clone(&self) -> GenerationOptions
fn clone(&self) -> GenerationOptions
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more