Skip to main content

Module postings

Module postings 

Source
Expand description

Posting-block decode kernels: bit-unpack and delta prefix sum. Posting-block decode kernels: bit-unpack and delta prefix sum.

§Why this is novel territory

No published Lucene, tantivy, PISA, or PEF postings measurement exists on Apple M-series silicon (research/02a:287, :516); tantivy’s SIMD decode is SSE2-gated and x86-only. A NEON block-max decode path on a 128-byte cache-line machine is ground nobody has taken.

§Shape

Two kernels, both following the Task 03/04 convention: scalar is the behavioural oracle, the NEON arm sits behind runtime feature detection, and prop_neon_unpack_equals_scalar proves equivalence over every bit width rather than over a sampled few.

  • bit-unpack widens a packed run into u32 lanes. The NEON arm uses only stable core::arch::aarch64 intrinsics — shifts, ands, and loads. Unlike sdot and fcvtl (Task 03), no inline assembly is needed here.
  • prefix sum turns docid gaps back into absolute ids. The shape is Lemire’s interleaved load/store form, which reaches 8.9 B values/s on an M4 against 3.9 B scalar.

No timing is recorded here. The decode-rate figure is NOT YET MEASURED: this machine is not single-tenant, and a contaminated number is worse than no number.

Constants§

NARROW_MAX_BITS
Unpacks using the best available arm.

Functions§

prefix_sum
Runs the prefix sum using the best available arm.
prefix_sum_scalar
Turns gaps into absolute ids in place, scalar oracle.
prefix_sum_wide
Turns gaps into absolute ids, sixteen values per group.
unpack
Runs the bit-unpack using the best available arm.
unpack_narrow
Unpacks at eight bits or fewer, eight values per iteration.
unpack_scalar
Unpacks count values of bits each, scalar oracle.