mcap2arrow

mcap2arrow is a Rust library that decodes MCAP channels/messages and exposes rows as Apache Arrow RecordBatch streams.

Installation

[dependencies]
mcap2arrow = "0.2.0"

What It Provides

Reader API over MCAP files and memory maps
Conversion from decoded message values to Arrow arrays/schema
Decoder registration API for different schema/message encodings
Built-in optional decoders via feature flags

Feature Flags

Default features:

protobuf
ros2msg
ros2idl

Disable defaults to trim dependencies:

[dependencies]
mcap2arrow = { version = "0.2.0", default-features = false, features = ["protobuf"] }

Encoding pairs supported by built-in decoders:

Schema encoding	Message encoding	Feature
`protobuf`	`protobuf`	`protobuf`
`ros2msg`	`cdr`	`ros2msg`
`ros2idl`	`cdr`	`ros2idl`

Minimal Usage

use std::path::Path;
use mcap2arrow::McapReader;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let reader = McapReader::builder().with_default_decoders().build();

    reader.for_each_record_batch(Path::new("sample.mcap"), "/topic/name", |batch| {
        println!("rows={}, cols={}", batch.num_rows(), batch.num_columns());
        Ok(())
    })?;

    Ok(())
}

Related Crates in This Workspace

mcap2arrow-core: schema/value model and shared errors
mcap2arrow-arrow: Arrow conversion implementation
mcap2arrow-protobuf: protobuf decoder
mcap2arrow-ros2msg: ROS 2 .msg decoder
mcap2arrow-ros2idl: ROS 2 IDL decoder

CLI

If you want a command-line interface, see transmcap: