Struct PendingTextRequest 

pub struct PendingTextRequest { /* private fields */ }

The fluent, chainable way to build and run a Text generation request.

Get one of these from Registry::text, chain .with_*() calls to describe what you want, then call generate to actually send the request (running the full multi-step tool-calling loop if you attached tools).

Example

use llmprism::Registry;

let registry = Registry::from_env();

let response = registry
    .text("openai", "gpt-4o-mini")?
    .with_system_prompt("You are a concise assistant.")
    .with_prompt("What's the capital of France?")
    .with_temperature(0.2)
    .with_max_tokens(100)
    .generate()
    .await?;

println!("{}", response.text.unwrap_or_default());

Implementations

impl PendingTextRequest

pub fn new(provider: Arc<dyn Provider>, model: impl Into<String>) -> Self

Starts a new builder for provider, targeting model. You’ll normally get one of these from Registry::text rather than calling this directly.

pub fn with_system_prompt(self, prompt: impl Into<String>) -> Self

Adds a system-level instruction – guidance for how the model should behave that isn’t part of the visible conversation (a persona, a style guide, ground rules). Call this more than once to add several; they’re combined in the order you added them.

pub fn with_prompt(self, prompt: impl Into<String>) -> Self

Adds a plain-text user message to the conversation. This is the shortcut for the common case of a single-turn question; for a multi-turn conversation, build it up with repeated calls to this method and/or with_message.

pub fn with_message(self, message: Message) -> Self

Appends one message (of any role) to the conversation. Use this when you need more control than with_prompt gives you – for example, replaying a stored conversation history.

pub fn with_messages(self, messages: impl IntoIterator<Item = Message>) -> Self

Appends several messages to the conversation at once, in order.

pub fn with_max_tokens(self, max_tokens: u32) -> Self

Caps how many tokens the model may generate in its reply.

pub fn with_temperature(self, temperature: f32) -> Self

Sets the sampling temperature. Typical values range from 0.0 (deterministic, focused) to around 1.0+ (more varied, creative) – check your chosen provider’s docs for its exact accepted range.

pub fn with_top_p(self, top_p: f32) -> Self

Sets the nucleus-sampling cutoff (top_p). Most applications only need to tune one of temperature or top_p, not both.

pub fn with_stop_sequence(self, stop: impl Into<String>) -> Self

Adds a string that stops generation the moment the model produces it. Call this more than once to add several. See TextRequest::stop_sequences for which providers support this.

pub fn with_seed(self, seed: u64) -> Self

Sets a seed for providers whose backend can honor one. See TextRequest::seed for what “honor” means here.

pub fn with_max_steps(self, max_steps: u32) -> Self

Sets how many request/response round trips the tool-calling loop is allowed to run before it stops on its own. If you’re attaching tools with with_tool, you’ll usually want this higher than the default of 1 (which only allows a single round trip, with no room for a tool call and a follow-up reply).

pub fn with_stop_when(self, condition: impl StopCondition + 'static) -> Self

Adds a condition that can end the tool-calling loop early, checked after every round trip alongside max_steps. Call this more than once to attach several – the loop stops as soon as any one of them (or max_steps, or the model simply not asking for another tool call) says to. See StopCondition.

pub fn with_tool(self, tool: impl Tool + 'static) -> Self

Attaches one tool the model may call during this request. Call this once per tool. See Tool for how to define one, and remember to raise with_max_steps above 1 so there’s room for the model to use the tool’s result in a follow-up reply.

pub fn with_tools(self, tools: impl IntoIterator<Item = Arc<dyn Tool>>) -> Self

Attaches several already-constructed tools at once – handy when you keep a shared Vec<Arc<dyn Tool>> of tools available to your application and want to reuse it across requests.

pub fn with_approval_handler( self, handler: impl ApprovalHandler + 'static, ) -> Self

Sets the handler consulted before running a tool call whose Tool::needs_approval returns true. Has no effect on tools that don’t opt into approval. See ApprovalHandler and TextRequest::approval_handler for what happens with no handler attached at all.

pub fn with_provider_tool(self, tool: Value) -> Self

Attaches one provider-native tool – a built-in, server-side capability like web search, in whatever JSON shape the target provider expects for it. See TextRequest::provider_tools for which providers support this at all.

use llmprism::providers::anthropic::AnthropicProvider;
use llmprism::Registry;
use serde_json::json;

let mut registry = Registry::new();
registry.register("anthropic", AnthropicProvider::new("sk-ant-..."));

let request = registry
    .text("anthropic", "claude-sonnet-4-5")
    .unwrap()
    .with_prompt("What's the latest Rust release?")
    .with_provider_tool(json!({
        "type": "web_search_20250305",
        "name": "web_search",
    }))
    .to_request();

pub fn with_tool_choice(self, choice: ToolChoice) -> Self

Controls how free the model is to decide whether to call a tool. See ToolChoice. Has no effect unless tools are attached.

pub fn with_prompt_caching(self) -> Self

Marks the system prompt as a cache breakpoint on providers that support explicit prompt caching (currently just Anthropic; a no-op elsewhere). See TextRequest::cache_system_prompt for when this is worth turning on.

pub fn with_thinking_budget(self, budget_tokens: u32) -> Self

Enables Claude’s extended thinking, capping its internal reasoning at budget_tokens. Anthropic requires max_tokens to be strictly greater than budget_tokens; if with_max_tokens wasn’t called (or was called with too small a value), this crate raises the value actually sent so the request doesn’t fail for a reason that isn’t obvious from the error alone. See TextRequest::thinking_budget for the structured-output caveat.

pub fn with_reasoning_effort(self, effort: impl Into<String>) -> Self

Sets how much effort a reasoning model spends thinking before replying. See TextRequest::reasoning_effort for which providers and models this applies to.

pub fn with_conversation_id(self, id: impl Into<String>) -> Self

Opts this request into persistence via a PersistenceMiddleware attached to the provider, under id. See TextRequest::conversation_id.

pub fn to_request(&self) -> TextRequest

Freezes the builder’s current state into a TextRequest without sending it. Mainly useful for inspecting or logging what would be sent, or for advanced cases where you want to drive crate::tool_loop::run_text yourself.

pub async fn generate(self) -> Result<TextResponse, Error>

Sends the request and returns the model’s final answer.

If tools are attached and the model asks to call one, this handles the entire back-and-forth automatically – running the tool, sending its result back, and continuing the conversation – until the model produces a final answer or with_max_steps is reached. See crate::tool_loop for the details.

pub fn stream(self) -> BoxStream<'static, Result<StreamEvent, Error>>

Sends the request and returns a stream of StreamEvents, instead of waiting for the model’s entire reply the way generate does. Useful for showing a reply as it’s generated rather than all at once.

Tool calling works the same way it does for generate: if the model asks to call a tool, this handles running it, reports the result as a StreamEvent::ToolResult, and continues streaming the model’s next round trip automatically. The stream ends with exactly one StreamEvent::StreamEnd, carrying the same final result generate would have returned.