Crate nv_perception 

nv-perception

Perception stage model and artifact types for the NextVision runtime.

This crate defines:

• Stage — the core user-implementable trait for per-frame processing.
• Detection, DetectionSet — per-frame object detections.
• Track, TrackObservation, TrackState — tracked objects.
• PerceptionArtifacts — accumulated outputs of all stages for a frame.
• DerivedSignal, SignalValue — generic named signals.
• SceneFeature, SceneFeatureValue — scene-level feature observations.
• StagePipeline — ordered pipeline builder with validate().
• StageCapabilities — declared stage inputs/outputs for validation.

Supported model patterns

The library supports multiple perception model architectures through the same Stage trait. A stage may produce any combination of output fields — the pipeline does not enforce a fixed detect→track sequence.

Pattern	Description
Classical detector → tracker	Separate detection and tracking stages in sequence.
Joint detection+tracking	Single stage producing both detections and tracks.
Direct track emitter	Model outputs tracks directly, no intermediate detections.
Richer observations	Per-observation metadata via TrackObservation::metadata.
Scene/temporal analysis	Stages producing SceneFeatures or DerivedSignals.

See the Stage trait documentation for detailed examples and the StageOutput docs for joint-model and direct-track-emitter patterns.

Ergonomic entity construction

Use Detection::builder() and Track::new() to avoid manual struct construction. StageOutput::build() provides an incremental builder for assembling multi-field outputs, and StageOutput::with_artifact() is a shorthand for typed artifact-only output.

Convenience re-exports

This crate re-exports types that appear in Stage trait signatures so that stage authors can import everything from a single crate: StageId, StageError, FeedId, ViewEpoch, ViewSnapshot, FrameEnvelope, TypedMetadata, and StageMetrics.

Stage execution model

Stages execute linearly and synchronously within a feed. For each frame:

1. A PerceptionArtifacts accumulator starts empty.
2. Each stage receives a StageContext with the frame, prior artifacts, temporal snapshot, and view snapshot.
3. Each stage returns a StageOutput that is merged into the accumulator.
4. After all stages run, the accumulator feeds the temporal store and output.

Stages take &mut self — they run on a dedicated thread per feed and are never shared across threads.

Single Stage trait

The library uses one trait for all stage types — detection, tracking, classification, scene analysis, etc. Specialization happens by convention (which StageOutput fields a stage populates), not by type hierarchy. The abstraction stays minimal, avoids taxonomy assumptions, and lets users compose arbitrary pipeline shapes.

Extension seam: typed artifacts

For inter-stage data that does not fit the built-in fields (feature maps, prepared clip/window tensors, calibration data), stages use the stage_artifacts type-map. A pre-processing stage can assemble a sliding window and store it as a typed artifact for a downstream inference stage to consume.

Backend independence

This crate has no dependency on GStreamer or any media backend. Stages receive FrameEnvelope (generic pixel data) and produce domain-agnostic artifacts.

Re-exports

pub use artifact::PerceptionArtifacts;

pub use batch::BatchEntry;

pub use batch::BatchProcessor;

pub use detection::Detection;

pub use detection::DetectionBuilder;

pub use detection::DetectionSet;

pub use pipeline::StagePipeline;

pub use pipeline::StagePipelineBuilder;

pub use pipeline::ValidationMode;

pub use pipeline::ValidationWarning;

pub use pipeline::validate_pipeline_phased;

pub use pipeline::validate_stages;

pub use scene::SceneFeature;

pub use scene::SceneFeatureValue;

pub use signal::DerivedSignal;

pub use signal::SignalValue;

pub use stage::Stage;

pub use stage::StageCapabilities;

pub use stage::StageCategory;

pub use stage::StageContext;

pub use stage::StageOutput;

pub use stage::StageOutputBuilder;

pub use temporal_access::TemporalAccess;

pub use track::Track;

pub use track::TrackObservation;

pub use track::TrackState;

Modules

artifact

Accumulated perception artifacts and stage output merging.

batch

Shared batch processing trait and entry types.

detection

Detection types — individual detections and per-frame detection sets.

pipeline

Stage pipeline composition — ordered collections with optional validation.

scene

Scene-level features — observations about the entire visible scene.

signal

Generic derived signals — named scalar/vector/boolean/categorical values.

stage

The Stage trait — the core user-implementable perception contract.

temporal_access

Typed temporal-access contract for stages.

track

Track types — tracked objects across frames.

Structs

FeedId

Unique identifier for a video feed within the runtime.

FrameEnvelope

An immutable, ref-counted video frame.

StageId

Identifier for a perception stage.

StageMetrics

Per-stage metrics snapshot, included in StageContext for stages that want to adapt behavior based on their own performance.

TypedMetadata

Typed metadata bag — stores arbitrary Send + Sync + 'static values keyed by their concrete type.

ViewEpoch

Opaque monotonic epoch counter.

ViewSnapshot

Read-only, cheaply-cloneable snapshot of ViewState.

Enums

StageError

Errors from perception stages.