Struct GenerationConfig 

pub struct GenerationConfig {
    pub temperature: Option<f64>,
    pub max_tokens: Option<i32>,
    pub top_p: Option<f64>,
    pub top_k: Option<i32>,
    pub stop_sequences: Option<Vec<String>>,
    pub json_mode: Option<bool>,
    pub json_schema: Option<Value>,
    pub max_tool_rounds: Option<usize>,
    pub prompt_caching: Option<bool>,
}

Configuration for text generation.

Every field is optional; unset fields are simply omitted from the provider request, so the provider default applies.

Examples

use rai_sdk::{GenerationConfig, JsonSchema};
use serde::Deserialize;

#[derive(Deserialize, JsonSchema)]
#[schemars(crate = "rai_sdk::schemars")]
struct Summary {
    headline: String,
    bullets: Vec<String>,
}

// Free-form generation.
let creative = GenerationConfig::new().with_temperature(0.9);

// Schema-constrained generation derived from a Rust type.
let strict = GenerationConfig::new()
    .with_temperature(0.0)
    .with_json_schema_for::<Summary>()?;

Fields

temperature: Option<f64>

Temperature for sampling (0.0 to 2.0 typically).

max_tokens: Option<i32>

Maximum number of tokens to generate.

top_p: Option<f64>

Top-p (nucleus) sampling.

top_k: Option<i32>

Top-k sampling (not supported by all providers).

stop_sequences: Option<Vec<String>>

Stop sequences.

json_mode: Option<bool>

Whether to request JSON output (provider-dependent).

json_schema: Option<Value>

JSON Schema for structured output (provider-dependent).

max_tool_rounds: Option<usize>

Maximum number of tool execution rounds before failing.

prompt_caching: Option<bool>

Whether Anthropic prompt caching is enabled.

Implementations

impl GenerationConfig

pub fn new() -> Self

Start with an empty config — add only the overrides you need.

pub fn with_temperature(self, temperature: f64) -> Self

Set the sampling temperature. Higher values are more random.

Ignored for OpenAI reasoning (o-series) models, which do not accept it.

pub fn with_max_tokens(self, max_tokens: i32) -> Self

Cap the number of tokens the model may generate.

pub fn with_top_p(self, top_p: f64) -> Self

Set nucleus (top-p) sampling.

Ignored for OpenAI reasoning (o-series) models.

pub fn with_top_k(self, top_k: i32) -> Self

Set top-k sampling. Only sent to providers that support it.

pub fn with_stop_sequences(self, stop_sequences: Vec<String>) -> Self

Stop generating as soon as one of these sequences is produced.

pub fn with_json_mode(self, json_mode: bool) -> Self

Ask the provider for syntactically valid JSON without constraining its shape.

A JSON schema set through GenerationConfig::with_json_schema or GenerationConfig::with_json_schema_for takes precedence over this flag.

pub fn with_json_schema(self, json_schema: Value) -> Self

Constrain the response with a hand-written JSON Schema.

Prefer GenerationConfig::with_json_schema_for when the shape is already expressed as a Rust type.

pub fn with_json_schema_for<T>(self) -> Result<Self>

where T: JsonSchema,

Generate a JSON Schema from a Rust type and normalize object schemas for strict structured-output providers.

The schema is generated with inline_subschemas = true and no top-level "$schema" key, so non-recursive nested types are inlined directly rather than emitting "$defs"/"$ref". This matters because Gemini’s generation_config.response_schema (reachable in this SDK through the OpenRouter provider) rejects schemas containing "$schema", "$defs", or "$ref" keys with a 400 INVALID_ARGUMENT error. See the crate-internal schema normalizer’s documentation for the limits of this approach with recursive types.

Errors

Returns Error::Serialization if the schema generated for T cannot be converted to a JSON value.

pub fn with_max_tool_rounds(self, max_tool_rounds: usize) -> Self

Limit how many request/tool-execution rounds a single generate() call may run before failing with Error::ToolLoopLimitExceeded.

pub fn with_prompt_caching(self, enabled: bool) -> Self

Enable or disable Anthropic prompt caching for this request.

When enabled, Anthropic requests mark the system prompt and the final tool definition as ephemeral cache breakpoints. Other providers silently ignore this setting.

Examples

use rai_sdk::GenerationConfig;

let config = GenerationConfig::new().with_prompt_caching(true);
assert_eq!(config.prompt_caching, Some(true));

pub fn tool_round_limit(&self) -> usize

The effective maximum number of tool-calling rounds (defaults to 8).

Trait Implementations

impl Clone for GenerationConfig

fn clone(&self) -> GenerationConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GenerationConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for GenerationConfig

fn default() -> GenerationConfig

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for GenerationConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for GenerationConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.