Struct InferenceStage 

pub struct InferenceStage {
    pub pools: Arc<InferencePools>,
    pub outcomes: UnboundedSender<InferenceOutcome>,
    pub transition_outcomes: UnboundedSender<InferenceOutcome>,
    pub compaction_outcomes: UnboundedSender<CompactionOutcome>,
    pub content_summary_outcomes: UnboundedSender<CompactionOutcome>,
    pub wake: Arc<Notify>,
    pub runtime: Handle,
    pub exact_token_counting: bool,
}

The plumbing the inference-dispatch system needs: the per-model pools, the channel to report outcomes on, the tick wake handle, and a runtime handle to spawn the (bounded, per-request) worker tasks onto.

Fields

pools: Arc<InferencePools>

Per-model concurrency pools.

outcomes: UnboundedSender<InferenceOutcome>

Where completed inferences are reported.

transition_outcomes: UnboundedSender<InferenceOutcome>

Where completed transition-choice inferences are reported (a separate lane so the collect systems don’t confuse a routing decision with a normal agent turn).

compaction_outcomes: UnboundedSender<CompactionOutcome>

Where completed compaction jobs (LLM context summarization) are reported - again a separate lane so a summary isn’t mistaken for a turn.

content_summary_outcomes: UnboundedSender<CompactionOutcome>

Where completed content-summary transform jobs are reported (the Summarize context-transform lane - see context_transform).

wake: Arc<Notify>

Signalled when an inference completes, to wake the tick loop.

runtime: Handle

Runtime the worker tasks are spawned onto.

exact_token_counting: bool

Opt-in: perform an exact pre-inference token count and reject requests that would overflow the model’s context window (see InferenceJob::exact_token_counting). Off by default.

Trait Implementations

impl Clone for InferenceStage

fn clone(&self) -> InferenceStage

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Component for InferenceStage

where Self: Send + Sync + 'static,

const STORAGE_TYPE: StorageType = bevy_ecs::component::StorageType::SparseSet

A constant indicating the storage type used for this component.

type Mutability = Mutable

A marker type to assist Bevy with determining if this component is mutable, or immutable. Mutable components will have Component<Mutability = Mutable>, while immutable components will instead have Component<Mutability = Immutable>.

fn register_required_components( _requiree: ComponentId, required_components: &mut RequiredComponentsRegistrator<'_, '_>, )

Registers required components.

fn clone_behavior() -> ComponentCloneBehavior

Called when registering this component, allowing to override clone function (or disable cloning altogether) for this component.

fn relationship_accessor() -> Option<ComponentRelationshipAccessor<Self>>

Returns ComponentRelationshipAccessor required for working with relationships in dynamic contexts.

fn on_add() -> Option<for<'w> fn(DeferredWorld<'w>, HookContext)>

Gets the on_add ComponentHook for this Component if one is defined.

fn on_insert() -> Option<for<'w> fn(DeferredWorld<'w>, HookContext)>

Gets the on_insert ComponentHook for this Component if one is defined.

fn on_discard() -> Option<for<'w> fn(DeferredWorld<'w>, HookContext)>

Gets the on_discard ComponentHook for this Component if one is defined.

fn on_remove() -> Option<for<'w> fn(DeferredWorld<'w>, HookContext)>

Gets the on_remove ComponentHook for this Component if one is defined.

fn on_despawn() -> Option<for<'w> fn(DeferredWorld<'w>, HookContext)>

Gets the on_despawn ComponentHook for this Component if one is defined.

fn map_entities<E>(_this: &mut Self, _mapper: &mut E)

where E: EntityMapper,

Maps the entities on this component using the given EntityMapper. This is used to remap entities in contexts like scenes and entity cloning. When deriving Component, this is populated by annotating fields containing entities with #[entities]

impl Resource for InferenceStage

where Self: Send + Sync + 'static,