scirs2-io 0.4.2

Input/Output utilities module for SciRS2 (scirs2-io)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

//! Lance-inspired columnar storage format (pure Rust, in-process).
//!
//! Provides reading and writing of a Lance-like binary columnar format:
//!
//! - `LanceSchema` / `LanceField` / `LanceDataType` — schema descriptions
//! - `LanceColumn` / `LanceBatch` — in-memory column storage
//! - `LanceWriter` — streaming batch writer (`Write` sink)
//! - `LanceReader` — batch reader (`Read + Seek` source)
//!
//! Binary layout:
//! ```text
//! File  = LANCE001(8) + SchemaLen(4 LE u32) + SchemaJSON(N) + Batch* + Footer
//! Batch = NumRows(4 LE u32) + Column*
//! Col   = TypeTag(1) + Nullable(1) + DataLen(4 LE u32) + Data + [validity]
//! Footer= BatchCount(4 LE u32) + LANCEEOF(8)
//! ```

pub mod types;
pub mod writer;

pub use types::{LanceBatch, LanceColumn, LanceDataType, LanceField, LanceSchema};
pub use writer::{LanceReader, LanceWriter, LANCE_EOF, LANCE_MAGIC};