decimal-bytes

Arbitrary precision decimals with lexicographically sortable byte encoding.

Overview

This crate provides three decimal types optimized for database storage:

• Decimal: Variable-length arbitrary precision (up to 131,072 digits)
• Decimal64: Fixed 8-byte representation with embedded scale (precision ≤ 16 digits)
• Decimal64NoScale: Fixed 8-byte representation with external scale (precision ≤ 18 digits)

All types support PostgreSQL special values (NaN, ±Infinity) with correct sort ordering.

Why not use rust_decimal or bigdecimal? Those libraries are excellent for arithmetic, but their byte representations are not lexicographically sortable. You cannot compare their serialized bytes to determine numerical order - you must deserialize first. decimal-bytes solves this by providing a byte encoding where bytes(a) < bytes(b) if and only if a < b numerically.

When to Use Which

Features

• Three storage options: Fixed 8-byte (Decimal64, Decimal64NoScale) or variable-length (Decimal)
• Columnar-friendly: Decimal64NoScale enables correct aggregates with external scale
• Lexicographic ordering: Byte comparison matches numerical comparison
• PostgreSQL NUMERIC compatibility: Full support for precision, scale (including negative), and special values
• Special values: Infinity, -Infinity, and NaN with correct PostgreSQL sort order

Decimal64 Usage

For most financial and business applications where precision ≤ 16 digits:

use decimal_bytes::Decimal64;

// Create with scale
let price = Decimal64::new("99.99", 2).unwrap();
assert_eq!(price.to_string(), "99.99");
assert_eq!(price.scale(), 2);

// Parse with automatic scale detection
let d: Decimal64 = "123.456".parse().unwrap();
assert_eq!(d.scale(), 3);

// Access raw components
let value = price.value();  // 9999 (scaled integer)
let scale = price.scale();  // 2

// Special values (PostgreSQL compatible)
let inf = Decimal64::infinity();
let neg_inf = Decimal64::neg_infinity();
let nan = Decimal64::nan();

// Correct sort order: -Infinity < numbers < +Infinity < NaN
assert!(neg_inf < price);
assert!(price < inf);
assert!(inf < nan);

// NaN equals NaN (PostgreSQL semantics)
assert_eq!(nan, Decimal64::nan());

Decimal64 with Precision and Scale (PostgreSQL NUMERIC)

Decimal64 fully supports PostgreSQL's NUMERIC(precision, scale) semantics:

use decimal_bytes::Decimal64;

// NUMERIC(5, 2) - up to 5 digits total, 2 after decimal
let d = Decimal64::with_precision_scale("123.456", Some(5), Some(2)).unwrap();
assert_eq!(d.to_string(), "123.46"); // Rounded to 2 decimal places

// Precision overflow - truncates from left (PostgreSQL behavior)
let d = Decimal64::with_precision_scale("12345.67", Some(5), Some(2)).unwrap();
assert_eq!(d.to_string(), "345.67"); // Keeps rightmost 5 digits

// NUMERIC(2, -3) - negative scale rounds to powers of 10
let d = Decimal64::with_precision_scale("12345", Some(2), Some(-3)).unwrap();
assert_eq!(d.to_string(), "12000"); // Rounded to nearest 1000

Decimal64 Storage Layout

64-bit packed representation:
┌──────────────────┬─────────────────────────────────────────────────────┐
│ Scale (8 bits)   │ Value (56 bits, signed)                             │
│ Byte 0           │ Bytes 1-7                                           │
└──────────────────┴─────────────────────────────────────────────────────┘

• Scale byte: 0-18 for normal values, 253/254/255 for -Infinity/+Infinity/NaN
• Value: 56-bit signed integer (-2^{55 to 2}55-1, ~16 significant digits)

Decimal64 Benefits

• Fixed 8 bytes: Predictable storage, no heap allocation, cache-friendly
• PostgreSQL compatible: Full NUMERIC(p,s) semantics including NaN, ±Infinity
• Fast operations: Single i64 comparison and serialization

Decimal64NoScale Usage (Recommended for Columnar Storage)

Decimal64NoScale stores the raw scaled value without embedding the scale, enabling:

• 18 digits of precision (vs 16 for Decimal64)
• Correct aggregates (SUM, MIN, MAX work directly on raw i64 values)
• Columnar storage compatibility (scale stored once in schema metadata)

use decimal_bytes::Decimal64NoScale;

// Scale is provided externally (e.g., from schema metadata)
let scale = 2;
let a = Decimal64NoScale::new("100.50", scale).unwrap();
let b = Decimal64NoScale::new("200.25", scale).unwrap();

// Raw values can be summed directly!
let sum = a.value() + b.value();  // 30075
assert_eq!(sum, 30075);

// Interpret result with scale
let result = Decimal64NoScale::from_raw(sum);
assert_eq!(result.to_string_with_scale(scale), "300.75");

// 18 digits supported (more than Decimal64's 16)
let big = Decimal64NoScale::new("123456789012345678", 0).unwrap();
assert_eq!(big.value(), 123456789012345678);

Why Decimal64NoScale for Aggregates?

Decimal64 embeds scale in the i64, which corrupts aggregate results:

Decimal64:        packed = (scale << 56) | mantissa
                  SUM(a, b) = adds scale bits → WRONG!

Decimal64NoScale: stored = value * 10^scale
                  SUM(a, b) = (a+b)*scale → divide by scale → CORRECT!

Decimal64NoScale Storage Layout

64-bit representation:
┌─────────────────────────────────────────────────────────────────┐
│ Value (64 bits, signed) - represents value * 10^scale           │
└─────────────────────────────────────────────────────────────────┘

• Value: Full 64-bit signed integer (±9.99×10^17, ~18 significant digits)
• Scale: Stored externally (e.g., in database schema)
• Special values: i64::MIN (NaN), i64::MIN+1 (-Infinity), i64::MAX (+Infinity)

Decimal Usage (Arbitrary Precision)

use decimal_bytes::Decimal;

// Create decimals from strings
let a = Decimal::from_str("123.456").unwrap();
let b = Decimal::from_str("123.457").unwrap();

// Byte comparison matches numerical comparison
assert!(a.as_bytes() < b.as_bytes());
assert!(a < b);

// With precision and scale constraints (SQL NUMERIC semantics)
let d = Decimal::with_precision_scale("123.456", Some(10), Some(2)).unwrap();
assert_eq!(d.to_string(), "123.46"); // Rounded to 2 decimal places

// Negative scale (rounds to left of decimal point)
let d = Decimal::with_precision_scale("12345", Some(10), Some(-3)).unwrap();
assert_eq!(d.to_string(), "12000"); // Rounded to nearest 1000

// Efficient byte access (primary representation)
let bytes: &[u8] = d.as_bytes();

// Reconstruct from bytes
let restored = Decimal::from_bytes(bytes).unwrap();
assert_eq!(d, restored);

Special Values

PostgreSQL-compatible special values with correct sort ordering:

use decimal_bytes::Decimal;

// Create special values
let pos_inf = Decimal::infinity();
let neg_inf = Decimal::neg_infinity();
let nan = Decimal::nan();

// Or parse from strings (case-insensitive)
let inf = Decimal::from_str("Infinity").unwrap();
let inf = Decimal::from_str("inf").unwrap();
let nan = Decimal::from_str("NaN").unwrap();

// Check for special values
assert!(pos_inf.is_infinity());
assert!(pos_inf.is_pos_infinity());
assert!(neg_inf.is_neg_infinity());
assert!(nan.is_nan());
assert!(!pos_inf.is_finite());

// Sort order: -Infinity < negatives < zero < positives < Infinity < NaN
assert!(neg_inf < Decimal::from_str("-1000000").unwrap());
assert!(Decimal::from_str("1000000").unwrap() < pos_inf);
assert!(pos_inf < nan);

PostgreSQL vs IEEE 754 Semantics

This library follows PostgreSQL semantics for special values, which differ from IEEE 754 floating-point:

use decimal_bytes::Decimal;

let nan1 = Decimal::nan();
let nan2 = Decimal::nan();
let inf = Decimal::infinity();

// NaN equals itself (PostgreSQL behavior, unlike IEEE 754)
assert_eq!(nan1, nan2);

// NaN is greater than everything, including Infinity
assert!(nan1 > inf);

This makes Decimal suitable for use in indexes, sorting, and deduplication where consistent ordering and equality semantics are required.

PostgreSQL Compatibility

This crate implements the PostgreSQL NUMERIC specification:

Storage Efficiency

The encoding matches PostgreSQL's storage efficiency (2 bytes per 4 decimal digits):

• 1 byte for sign
• 2 bytes for exponent
• ~N/2 bytes for N-digit mantissa (BCD encoding: 2 digits per byte)
• Special values: 3 bytes each

Example: A 9-digit number like 123456789 requires only ~8 bytes total.

Sort Order

The lexicographic byte order matches the PostgreSQL NUMERIC sort order:

-Infinity < negative numbers < zero < positive numbers < +Infinity < NaN

This enables efficient range queries in sorted key-value stores without decoding.

Performance

Type Comparison Summary

Memory Usage

Decimal64NoScale Operations (Recommended for Columnar)

Decimal64 Operations

Decimal Operations (Arbitrary Precision)

Aggregate Performance (Key Differentiator)

For columnar storage where aggregates are important:

Why is Decimal64NoScale faster?

• Decimal64NoScale.value() returns raw i64 directly
• Decimal64.value() must unpack/mask the 56-bit value from the packed format

Run cargo bench locally to reproduce benchmarks on your hardware.

Arithmetic Operations

This library focuses on storage and comparison, not arithmetic. Existing Rust decimal libraries (rust_decimal, bigdecimal) provide arithmetic but their byte representations are not lexicographically sortable - you cannot compare their serialized bytes to determine numerical order. That's the gap decimal-bytes fills: efficient storage with byte-level ordering for databases and search engines.

For calculations, use an established decimal library and convert:

With rust_decimal (recommended for most use cases)

[dependencies]
decimal-bytes = { version = "0.1", features = ["rust_decimal"] }

use rust_decimal::Decimal as RustDecimal;
use decimal_bytes::Decimal;

// Convert from rust_decimal for storage
let rd = RustDecimal::new(12345, 2); // 123.45
let stored: Decimal = rd.try_into().unwrap();

// Do arithmetic with rust_decimal
let a: RustDecimal = (&stored).try_into().unwrap();
let b = RustDecimal::new(1000, 2); // 10.00
let sum = a + b; // 133.45

// Convert back for storage
let result: Decimal = sum.try_into().unwrap();

With bigdecimal (for arbitrary precision arithmetic)

[dependencies]
decimal-bytes = { version = "0.1", features = ["bigdecimal"] }

use bigdecimal::BigDecimal;
use decimal_bytes::Decimal;
use std::str::FromStr;

// Convert between types
let bd = BigDecimal::from_str("123.456789012345678901234567890").unwrap();
let stored: Decimal = bd.try_into().unwrap();
let restored: BigDecimal = (&stored).try_into().unwrap();

License

MIT License - see LICENSE for details.