Module format 

EdgeFirst Dataset Format utilities.

This module provides tools for working with the EdgeFirst Dataset Format as documented in DATASET_FORMAT.md. It enables:

• Reading and resolving file paths from Arrow annotation files
• Generating Arrow files from folders of images (with null annotations)
• Validating dataset directory structures
• (Future) Converting from other formats (COCO, DarkNet, YOLO, etc.)

EdgeFirst Dataset Format

A dataset in EdgeFirst format consists of:

• An Arrow file ({dataset_name}.arrow) containing annotation metadata
• A sensor container directory ({dataset_name}/) with image/sensor files

Supported Structures

Sequence-based (frame column is not null):

dataset_name/
├── dataset_name.arrow
└── dataset_name/
    └── sequence_name/
        ├── sequence_name_001.camera.jpeg
        └── sequence_name_002.camera.jpeg

Image-based (frame column is null):

dataset_name/
├── dataset_name.arrow
└── dataset_name/
    ├── image1.jpg
    └── image2.png

Example

use edgefirst_client::format::{resolve_arrow_files, validate_dataset_structure};
use std::path::Path;

// Resolve all files referenced by an Arrow file
let arrow_path = Path::new("my_dataset/my_dataset.arrow");
let files = resolve_arrow_files(arrow_path)?;
for (name, path) in &files {
    println!("{}: {:?}", name, path);
}

// Validate the dataset structure
let issues = validate_dataset_structure(Path::new("my_dataset"))?;
if !issues.is_empty() {
    for issue in &issues {
        eprintln!("Warning: {}", issue);
    }
}

Structs

ResolvedFile

Information about a resolved sample file.

Enums

ValidationIssue

Validation issue found in dataset structure.

Constants

IMAGE_EXTENSIONS

Image file extensions supported by EdgeFirst.

Functions

generate_arrow_from_folder

Generate an Arrow file from a folder of images.

get_arrow_path

Get the expected Arrow file path for a dataset directory.

get_sensor_container_path

Get the expected sensor container path for a dataset directory.

resolve_arrow_files

Resolve all file paths referenced by an Arrow annotation file.

resolve_files_with_container

Resolve Arrow file references against actual files in a sensor container.

validate_dataset_structure

Validate the structure of a dataset directory.