Expand description
§Babbel Parquet
Fast, pure-Rust columnar reader, writer, and [FormatEngine] implementation for
Apache Parquet (.parquet), the industry-standard columnar big data format.
§Features
- Pure Rust: Zero external dependencies, self-contained Apache Thrift Compact Protocol codec.
- Standard Parquet Encoding: Valid
PAR1magic bytes, standardFileMetaData,RowGroup, andColumnChunkstructures. - Rich Types: Integers (
INT32,INT64), Floating-point (FLOAT,DOUBLE), Strings (BYTE_ARRAYUTF-8), Booleans, and Nullability. - Unified Architecture: Integrates into Babbel’s universal
ValueAST and cross-format conversion matrix.
§Quickstart
use babbel_core::Value;
use babbel_parquet::{read_parquet, write_parquet};
// Create tabular data as an Array of row Objects
let table = Value::Array(vec![
Value::Object(vec![
("id".into(), Value::Integer(1)),
("name".into(), Value::String("Alice".into())),
("score".into(), Value::Float(98.5)),
]),
Value::Object(vec![
("id".into(), Value::Integer(2)),
("name".into(), Value::String("Bob".into())),
("score".into(), Value::Float(85.0)),
]),
]);
// Write to standard Parquet bytes
let parquet_bytes = write_parquet(&table).expect("write failed");
assert_eq!(&parquet_bytes[0..4], b"PAR1");
// Read back into universal Value AST
let roundtrip = read_parquet(&parquet_bytes).expect("read failed");
assert_eq!(roundtrip, table);Re-exports§
pub use column::ColumnData;pub use engine::ParquetEngine;pub use error::ParquetError;pub use metadata::CompressionCodec;pub use reader::ParquetReadOptions;pub use reader::read_parquet;pub use reader::read_parquet_with_options;pub use writer::ParquetWriteOptions;pub use writer::write_parquet;pub use writer::write_parquet_with_options;
Modules§
- column
- Columnar data encoding and decoding for Parquet PLAIN and RLE_DICTIONARY formats.
- engine
- Parquet Format Engine implementation adhering to OCP and DIP.
- error
- Error types for Parquet columnar decoding and encoding.
- metadata
- Parquet schema, metadata structures, and Thrift codec.
- reader
- Apache Parquet file reader and row materializer.
- thrift
- Pure-Rust Thrift Compact Protocol encoder and decoder for Parquet metadata.
- writer
- Apache Parquet file writer and schema inference engine.
Enums§
- Value
- Universal hierarchical data value across XML, JSON, YAML, and Bencode representations.