tauq 0.2.0

Token-efficient data notation - 49% fewer tokens than JSON (verified with tiktoken)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

//! TBF-Iceberg Integration
//!
//! This module provides integration between TBF (Tauq Binary Format) and Apache Iceberg,
//! enabling TBF to be used as a file format for Iceberg tables.
//!
//! # Features
//!
//! - **TbfFileWriter**: Implements Iceberg's `FileWriter` trait for TBF format
//! - **Arrow conversion**: Converts Arrow RecordBatch to TBF columnar encoding
//! - **Statistics tracking**: Collects column statistics (min/max, null counts) during writes
//!
//! # Example
//!
//! ```rust,ignore
//! use tauq::tbf_iceberg::{TbfWriterBuilder, TbfWriter};
//! use iceberg::writer::file_writer::FileWriterBuilder;
//!
//! let builder = TbfWriterBuilder::new(schema);
//! let writer = builder.build(output_file).await?;
//! writer.write(&record_batch).await?;
//! let data_files = writer.close().await?;
//! ```

mod arrow_convert;
mod writer;

pub use arrow_convert::{ArrowToTbf, arrow_schema_to_tbf};
pub use writer::{TbfFileWriter, TbfFileWriterBuilder};