buff_rs/lib.rs
1//! # buff-rs
2//!
3//! A Rust implementation of BUFF: Decomposed Bounded Floats for Fast Compression and Queries.
4//!
5//! Based on the VLDB 2021 paper: ["BUFF: Accelerating Queries in Memory through
6//! Decomposed Bounded Floats"](https://dl.acm.org/doi/abs/10.14778/3476249.3476305).
7//!
8//! ## Overview
9//!
10//! BUFF provides efficient compression and query execution for bounded floating-point
11//! data (data with a known precision/scale). It achieves this through:
12//!
13//! 1. **Precision Bounding**: Determining the minimum bits needed to represent values
14//! within a given precision tolerance
15//! 2. **Byte Slicing**: Column-oriented storage where each bit position across all values
16//! is stored contiguously
17//! 3. **Direct Query Execution**: Aggregate and filter operations can execute directly
18//! on compressed data
19//!
20//! ## Key Differences from `decimal-bytes`
21//!
22//! | Feature | decimal-bytes | buff-rs |
23//! |---------|--------------|---------|
24//! | Data Type | Single decimal values | Arrays of floats |
25//! | Precision | Arbitrary (unlimited) | Bounded (fixed scale) |
26//! | Storage | Row-oriented | Column-oriented (byte-sliced) |
27//! | Primary Use | Document storage | Columnar/time-series data |
28//! | Queries | Compare after decode | Query compressed data |
29//!
30//! ## Quick Start
31//!
32//! ```rust
33//! use buff_rs::BuffCodec;
34//!
35//! // Create a codec with 3 decimal places of precision (scale=1000)
36//! let codec = BuffCodec::new(1000);
37//!
38//! // Encode an array of f64 values
39//! let data = vec![1.234, 5.678, 9.012, -3.456];
40//! let encoded = codec.encode(&data).unwrap();
41//!
42//! // Decode back to f64
43//! let decoded = codec.decode(&encoded).unwrap();
44//!
45//! // Query directly on compressed data
46//! let sum = codec.sum(&encoded).unwrap();
47//! let max = codec.max(&encoded).unwrap();
48//! ```
49//!
50//! ## Choosing a Scale
51//!
52//! The scale determines the precision of encoded values:
53//!
54//! | Scale | Decimal Places | Example |
55//! |-------|---------------|---------|
56//! | 10 | 1 | 3.1 |
57//! | 100 | 2 | 3.14 |
58//! | 1000 | 3 | 3.142 |
59//! | 10000 | 4 | 3.1416 |
60//! | 100000 | 5 | 3.14159 |
61//!
62//! Choose a scale that matches your data's required precision. Higher scales
63//! provide more precision but may reduce compression ratio.
64//!
65//! ## Compression Performance
66//!
67//! Compression ratio depends on data characteristics:
68//!
69//! - **Narrow range**: Better compression (fewer bits needed)
70//! - **Wide range**: More bits needed, lower compression
71//! - **Repetitive values**: Good compression
72//!
73//! Typical compression ratios range from 0.3x to 0.8x of the original size.
74
75#![warn(missing_docs)]
76#![warn(clippy::all)]
77#![allow(dead_code)]
78
79mod bitpack;
80mod codec;
81mod error;
82pub mod precision;
83
84#[cfg(feature = "decimal")]
85mod decimal_interop;
86
87pub use codec::{BuffCodec, BuffMetadata, SpecialValue, SpecialValueKind};
88pub use error::BuffError;
89
90#[cfg(feature = "decimal")]
91pub use decimal_interop::{Decimal64ArrayExt, DecimalArrayExt};
92
93/// Convenience type alias for Results with BuffError.
94pub type Result<T> = std::result::Result<T, BuffError>;