Skip to main content

Crate babbel_parquet

Crate babbel_parquet 

Source
Expand description

§Babbel Parquet

Fast, pure-Rust columnar reader, writer, and [FormatEngine] implementation for Apache Parquet (.parquet), the industry-standard columnar big data format.

§Features

  • Pure Rust: Zero external dependencies, self-contained Apache Thrift Compact Protocol codec.
  • Standard Parquet Encoding: Valid PAR1 magic bytes, standard FileMetaData, RowGroup, and ColumnChunk structures.
  • Rich Types: Integers (INT32, INT64), Floating-point (FLOAT, DOUBLE), Strings (BYTE_ARRAY UTF-8), Booleans, and Nullability.
  • Unified Architecture: Integrates into Babbel’s universal Value AST and cross-format conversion matrix.

§Quickstart

use babbel_core::Value;
use babbel_parquet::{read_parquet, write_parquet};

// Create tabular data as an Array of row Objects
let table = Value::Array(vec![
    Value::Object(vec![
        ("id".into(), Value::Integer(1)),
        ("name".into(), Value::String("Alice".into())),
        ("score".into(), Value::Float(98.5)),
    ]),
    Value::Object(vec![
        ("id".into(), Value::Integer(2)),
        ("name".into(), Value::String("Bob".into())),
        ("score".into(), Value::Float(85.0)),
    ]),
]);

// Write to standard Parquet bytes
let parquet_bytes = write_parquet(&table).expect("write failed");
assert_eq!(&parquet_bytes[0..4], b"PAR1");

// Read back into universal Value AST
let roundtrip = read_parquet(&parquet_bytes).expect("read failed");
assert_eq!(roundtrip, table);

Re-exports§

pub use column::ColumnData;
pub use engine::ParquetEngine;
pub use error::ParquetError;
pub use metadata::CompressionCodec;
pub use reader::ParquetReadOptions;
pub use reader::read_parquet;
pub use reader::read_parquet_with_options;
pub use writer::ParquetWriteOptions;
pub use writer::write_parquet;
pub use writer::write_parquet_with_options;

Modules§

column
Columnar data encoding and decoding for Parquet PLAIN and RLE_DICTIONARY formats.
engine
Parquet Format Engine implementation adhering to OCP and DIP.
error
Error types for Parquet columnar decoding and encoding.
metadata
Parquet schema, metadata structures, and Thrift codec.
reader
Apache Parquet file reader and row materializer.
thrift
Pure-Rust Thrift Compact Protocol encoder and decoder for Parquet metadata.
writer
Apache Parquet file writer and schema inference engine.

Enums§

Value
Universal hierarchical data value across XML, JSON, YAML, and Bencode representations.