Skip to main content

Module smallfloat

Module smallfloat 

Source
Expand description

SmallFloat: 1-byte quantized document lengths for BM25 fieldnorms.

Maps u32 doc lengths to a monotonic 0..255 byte scale.

  • Byte 0 → length 0
  • Byte 255 → length ~16M+
  • Monotonic: larger length → larger (or equal) byte
  • Max relative error dampened by BM25’s b=0.75 length normalization
  • 4x space reduction vs raw u32

Functions§

decode
Decode a byte back to an approximate u32 value.
encode
Encode a u32 value into a single byte.