Crate bevy_sensor 

bevy-sensor: Multi-view rendering for YCB object dataset

This library provides Bevy-based rendering of 3D objects from multiple viewpoints, designed to match TBP (Thousand Brains Project) habitat sensor conventions for use in neocortx sensorimotor learning experiments.

Headless Rendering (NEW)

Render directly to memory buffers for use in sensorimotor learning:

use bevy_sensor::{render_to_buffer, RenderConfig, ViewpointConfig, ObjectRotation};
use std::path::Path;

let config = RenderConfig::tbp_default(); // 64x64, RGBD
let viewpoint = bevy_sensor::generate_viewpoints(&ViewpointConfig::default())[0];
let rotation = ObjectRotation::identity();

let output = render_to_buffer(
    Path::new("/tmp/ycb/003_cracker_box"),
    &viewpoint,
    &rotation,
    &config,
)?;

// output.rgba: Vec<u8> - RGBA pixels (64*64*4 bytes)
// output.depth: Vec<f32> - Depth values (64*64 floats)

File-based Capture (Legacy)

use bevy_sensor::{SensorConfig, ViewpointConfig, ObjectRotation};

let config = SensorConfig {
    viewpoints: ViewpointConfig::default(),
    object_rotations: ObjectRotation::tbp_benchmark_rotations(),
    ..Default::default()
};

YCB Dataset

Download YCB models programmatically:

use bevy_sensor::ycb::{download_models, Subset};

// Download representative subset (3 objects)
download_models("/tmp/ycb", Subset::Representative).await?;

Re-exports

pub use batch::BatchRenderConfig;

pub use batch::BatchRenderError;

pub use batch::BatchRenderOutput;

pub use batch::BatchRenderRequest;

pub use batch::BatchRenderer;

pub use batch::BatchState;

pub use batch::RenderStatus;

pub use ycbust;

Modules

backend

WebGPU and cross-platform backend support for rendering.

batch

Batch rendering API for multiple viewpoints and objects.

cache

Model caching system for efficient multi-viewpoint rendering.

fixtures

Test fixtures for pre-rendered YCB images

ycb

YCB dataset utilities

Structs

CameraIntrinsics

Camera intrinsic parameters for 3D reconstruction.

CaptureCamera

Marker component for the capture camera

CaptureTarget

Marker component for the target object being captured

DownloadOptions

Options for downloading YCB objects.

LightingConfig

Lighting configuration for rendering.

ObjectRotation

Object rotation in Euler angles (degrees), matching TBP benchmark format. Format: [pitch, yaw, roll] or [x, y, z] rotation.

PersistentRenderer

Per-step persistent renderer for feedback loops. See the module docs in render::PersistentRenderer for lifetime, thread-affinity, and object/config-invariance guarantees. Built for the surface-policy use case in neocortx where a fixed object is rendered from a moving camera many times per episode (issue #65). Persistent per-step renderer. Loads the scene once at new() and renders one frame per render() call by mutating the camera transform and scene root rotation in-place. Built for surface-policy feedback loops where the object stays fixed for the duration of an episode and the camera moves every step. See issue #65.

Quat

A quaternion representing an orientation.

RenderConfig

Configuration for headless rendering.

RenderOutput

Output from headless rendering containing RGBA and depth data.

RenderSession

Persistent batch render session. See the module docs in render::RenderSession for lifetime, thread-affinity, and config-invariance guarantees. Persistent batch render session. Keeps a Bevy App (and its RenderDevice plus PSO cache) alive across multiple render() calls, amortizing per-episode cold-init cost.

SensorConfig

Full sensor configuration for capture sessions

Transform

Describe the position of an entity. If the entity has a parent, the position is relative to its parent position.

Vec3

A 3-dimensional vector.

ViewpointConfig

Configuration for viewpoint generation matching TBP habitat sensor behavior. Uses spherical coordinates to capture objects from multiple elevations.

Enums

RenderError

Errors that can occur during rendering and file operations.

YcbSubset

Subset of objects to download.

Constants

REPRESENTATIVE_OBJECTS

Representative subset of 3 commonly used objects.

TBP_SIMILAR_OBJECTS

TBP similar 10-object benchmark set (harder — similar geometry).

TBP_STANDARD_OBJECTS

TBP standard 10-object benchmark set (distinct objects).

Functions

create_batch_renderer

Create a new batch renderer helper for multi-viewpoint workflows.

generate_viewpoints

Generate camera viewpoints using spherical coordinates.

initialize

Initialize bevy-sensor rendering backend configuration.

queue_render_request

Queue a render request for batch processing.

render_all_viewpoints

Render all viewpoints and rotations for a YCB object.

render_batch

Render multiple requests in batch (convenience function).

render_next_in_batch

Process and execute the next render in the batch queue.

render_to_buffer

Render a YCB object to an in-memory buffer.

render_to_buffer_cached

Render with model caching support for efficient multi-viewpoint rendering.

render_to_files

Render directly to files (for subprocess mode).