pub struct FunctionInvocationConfig {
pub enabled: bool,
pub max_iterations: usize,
pub max_function_calls: Option<usize>,
pub max_duration_seconds: Option<f64>,
pub max_consecutive_errors_per_request: usize,
pub terminate_on_unknown_calls: bool,
pub include_detailed_errors: bool,
pub allow_concurrent_invocation: bool,
}Expand description
Configuration for the automatic function-invocation loop.
Mirrors FunctionInvocationConfiguration.
§The three bounds are complementary
max_iterations caps model round trips,
max_function_calls caps how many tool bodies
run in total however they are distributed across those trips, and
max_duration_seconds caps elapsed wall
time. A run that alternates one cheap call per iteration is bounded by the
first; one that emits twenty parallel calls per iteration by the second;
one whose tools are slow, or that waits on a human approval, by the third.
Fields§
§enabled: bool§max_iterations: usize§max_function_calls: Option<usize>Total tool executions allowed per request, cumulative across
iterations and across approval round trips. None (the default) is
unbounded.
Best-effort, checked after each batch completes: a model that
requests twenty parallel calls when four remain in the budget runs all
twenty, and the loop stops afterwards. Reaching the limit does not
fail the run — tools are disabled (ToolMode::None) and the model
is asked to answer with what it has, so the caller gets an answer
rather than an error.
max_duration_seconds: Option<f64>Wall-clock budget for the whole invocation loop, in seconds. None
(the default) is unbounded.
Measured from the loop’s start and checked after each batch, with the
same graceful degradation as
max_function_calls. Time spent waiting
for a human approval counts, deliberately: that is what bounds an
unattended run whose approvals nobody answers.
max_consecutive_errors_per_request: usize§terminate_on_unknown_calls: bool§include_detailed_errors: bool§allow_concurrent_invocation: boolWhether the executable calls of one model response may run
concurrently. true (the default) is what a model asking for parallel
calls is asking for; false runs them one at a time, in the order the
model emitted them.
Sequencing is not only about ordering the results — those come back in model order either way. It is about the side effects: a tool that writes to a shared resource, holds a non-reentrant handle, or is rate limited per caller behaves differently when two of its invocations overlap, and a model emitting “create the record, then update it” in one batch means them in that order.
One consequence worth stating, because it is the reason to want this: with concurrency off, a call that fails the batch (a middleware refusal) stops it, so the calls after it never run at all. With concurrency on, they were already in flight and are cancelled instead.
Mirrors upstream’s allow_concurrent_invocation (#8453).
Implementations§
Trait Implementations§
Source§impl Clone for FunctionInvocationConfig
impl Clone for FunctionInvocationConfig
Source§fn clone(&self) -> FunctionInvocationConfig
fn clone(&self) -> FunctionInvocationConfig
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more