Crate bevy_sensor 

bevy-sensor: Multi-view rendering for YCB object dataset

This library provides Bevy-based rendering of 3D objects from multiple viewpoints, designed to match TBP (Thousand Brains Project) habitat sensor conventions for use in neocortx sensorimotor learning experiments.

Headless Rendering (NEW)

Render directly to memory buffers for use in sensorimotor learning:

use bevy_sensor::{render_to_buffer, RenderConfig, ViewpointConfig, ObjectRotation};
use std::path::Path;

let config = RenderConfig::tbp_default(); // 64x64, RGBD
let viewpoint = bevy_sensor::generate_viewpoints(&ViewpointConfig::default())[0];
let rotation = ObjectRotation::identity();

let output = render_to_buffer(
    Path::new("/tmp/ycb/003_cracker_box"),
    &viewpoint,
    &rotation,
    &config,
)?;

// output.rgba: Vec<u8> - RGBA pixels (64*64*4 bytes)
// output.depth: Vec<f32> - Depth values (64*64 floats)

File-based Capture (Legacy)

use bevy_sensor::{SensorConfig, ViewpointConfig, ObjectRotation};

let config = SensorConfig {
    viewpoints: ViewpointConfig::default(),
    object_rotations: ObjectRotation::tbp_benchmark_rotations(),
    ..Default::default()
};

YCB Dataset

Download YCB models programmatically:

use bevy_sensor::ycb::{download_models, Subset};

// Download representative subset (3 objects)
download_models("/tmp/ycb", Subset::Representative).await?;

Re-exports

pub use batch::BatchRenderConfig;

pub use batch::BatchRenderError;

pub use batch::BatchRenderOutput;

pub use batch::BatchRenderRequest;

pub use batch::BatchRenderer;

pub use batch::BatchState;

pub use batch::RenderStatus;

pub use ycbust;

Modules

batch

Batch rendering API for multiple viewpoints and objects.

fixtures

Test fixtures for pre-rendered YCB images

ycb

YCB dataset utilities

Structs

CameraIntrinsics

Camera intrinsic parameters for 3D reconstruction.

CaptureCamera

Marker component for the capture camera

CaptureTarget

Marker component for the target object being captured

DownloadOptions

Options for downloading YCB objects.

LightingConfig

Lighting configuration for rendering.

ObjectRotation

Object rotation in Euler angles (degrees), matching TBP benchmark format. Format: [pitch, yaw, roll] or [x, y, z] rotation.

Quat

A quaternion representing an orientation.

RenderConfig

Configuration for headless rendering.

RenderOutput

Output from headless rendering containing RGBA and depth data.

SensorConfig

Full sensor configuration for capture sessions

Transform

Describe the position of an entity. If the entity has a parent, the position is relative to its parent position.

Vec3

A 3-dimensional vector.

ViewpointConfig

Configuration for viewpoint generation matching TBP habitat sensor behavior. Uses spherical coordinates to capture objects from multiple elevations.

Enums

RenderError

Errors that can occur during rendering and file operations.

YcbSubset

Subset of objects to download.

Constants

REPRESENTATIVE_OBJECTS

Representative subset of 3 commonly used objects.

TEN_OBJECTS

Subset of 10 commonly used objects.

Functions

create_batch_renderer

Create a new batch renderer for efficient multi-viewpoint rendering.

generate_viewpoints

Generate camera viewpoints using spherical coordinates.

queue_render_request

Queue a render request for batch processing.

render_all_viewpoints

Render all viewpoints and rotations for a YCB object.

render_batch

Render multiple requests in batch (convenience function).

render_next_in_batch

Process and execute the next render in the batch queue.

render_to_buffer

Render a YCB object to an in-memory buffer.

render_to_files

Render directly to files (for subprocess mode).