Skip to main content

FunctionInvocationConfig

Struct FunctionInvocationConfig 

Source
pub struct FunctionInvocationConfig {
    pub enabled: bool,
    pub max_iterations: usize,
    pub max_function_calls: Option<usize>,
    pub max_duration_seconds: Option<f64>,
    pub max_consecutive_errors_per_request: usize,
    pub terminate_on_unknown_calls: bool,
    pub include_detailed_errors: bool,
    pub allow_concurrent_invocation: bool,
}
Expand description

Configuration for the automatic function-invocation loop.

Mirrors FunctionInvocationConfiguration.

§The three bounds are complementary

max_iterations caps model round trips, max_function_calls caps how many tool bodies run in total however they are distributed across those trips, and max_duration_seconds caps elapsed wall time. A run that alternates one cheap call per iteration is bounded by the first; one that emits twenty parallel calls per iteration by the second; one whose tools are slow, or that waits on a human approval, by the third.

Fields§

§enabled: bool§max_iterations: usize§max_function_calls: Option<usize>

Total tool executions allowed per request, cumulative across iterations and across approval round trips. None (the default) is unbounded.

Best-effort, checked after each batch completes: a model that requests twenty parallel calls when four remain in the budget runs all twenty, and the loop stops afterwards. Reaching the limit does not fail the run — tools are disabled (ToolMode::None) and the model is asked to answer with what it has, so the caller gets an answer rather than an error.

§max_duration_seconds: Option<f64>

Wall-clock budget for the whole invocation loop, in seconds. None (the default) is unbounded.

Measured from the loop’s start and checked after each batch, with the same graceful degradation as max_function_calls. Time spent waiting for a human approval counts, deliberately: that is what bounds an unattended run whose approvals nobody answers.

§max_consecutive_errors_per_request: usize§terminate_on_unknown_calls: bool§include_detailed_errors: bool§allow_concurrent_invocation: bool

Whether the executable calls of one model response may run concurrently. true (the default) is what a model asking for parallel calls is asking for; false runs them one at a time, in the order the model emitted them.

Sequencing is not only about ordering the results — those come back in model order either way. It is about the side effects: a tool that writes to a shared resource, holds a non-reentrant handle, or is rate limited per caller behaves differently when two of its invocations overlap, and a model emitting “create the record, then update it” in one batch means them in that order.

One consequence worth stating, because it is the reason to want this: with concurrency off, a call that fails the batch (a middleware refusal) stops it, so the calls after it never run at all. With concurrency on, they were already in flight and are cancelled instead.

Mirrors upstream’s allow_concurrent_invocation (#8453).

Implementations§

Trait Implementations§

Source§

impl Clone for FunctionInvocationConfig

Source§

fn clone(&self) -> FunctionInvocationConfig

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for FunctionInvocationConfig

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more
Source§

impl Default for FunctionInvocationConfig

Source§

fn default() -> FunctionInvocationConfig

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more