Module simd_filter 

SIMD Vectorized Query Filters

From mm.md Task 5.3: AVX-512/NEON Vectorized Filters (VQE)

Problem

Current filtering is scalar. LLM context queries often filter millions of rows (e.g., “events from last 7 days”). SIMD can evaluate 8-16 predicates per instruction.

Solution

Column-oriented data layout + compiled filter expressions + runtime SIMD feature detection

Throughput Analysis

Scalar: 1 comparison/cycle × 3GHz = 3B comparisons/sec
AVX-512: 8 comparisons/instruction × ~1 CPI × 3GHz = 24B/sec
AVX-256: 4 comparisons/instruction = 12B/sec
NEON: 4 comparisons/instruction = 12B/sec

100M rows @ 24B/sec = 4.2ms (AVX-512)
100M rows @ 3B/sec = 33ms (scalar)

Speedup: 8× (AVX-512), 4× (AVX-256/NEON)

Structs

SimdInfo

SIMD capability information

Enums

FilterOp

Filter comparison operation

Functions

allocate_bitmap

Allocate a bitmap for the given number of rows

bitmap_and

AND two bitmaps together

bitmap_not

NOT a bitmap

bitmap_or

OR two bitmaps together

filter_f64_gt

Filter f64 column: value > threshold

filter_f64_gt_scalar

Scalar filter: f64 > threshold

filter_f64_lt_scalar

Scalar filter: f64 < threshold

filter_i64_between

Filter i64 column: low <= value <= high

filter_i64_between_scalar

Scalar filter: i64 between low and high (inclusive)

filter_i64_eq

Filter i64 column: value == target

filter_i64_eq_scalar

Scalar filter: i64 == value

filter_i64_ge_scalar

Scalar filter: i64 >= threshold

filter_i64_gt

Filter i64 column: value > threshold

filter_i64_gt_scalar

Scalar filter: i64 > threshold

filter_i64_lt

Filter i64 column: value < threshold

filter_i64_lt_scalar

Scalar filter: i64 < threshold

get_bit

Check if a bit is set

popcount

Count set bits in bitmap

set_bit

Set a bit in the bitmap

simd_info

Get information about SIMD support

Type Aliases

FilterBitmap

Result bitmap - bit per row indicating pass/fail