Module hot_path_layout 

Vector Storage Hot-Path Layout (Task 16)

Optimized memory layout for vector storage focused on hot-path performance:

• Contiguous embedding storage for cache efficiency
• SIMD-aligned vectors ┌─────────────────────────────────────────────────────────────────┐ │ Embedding Block (64KB aligned) │ ├─────────────────────────────────────────────────────────────────┤ │ Header (64B) │ Padding │ Vectors (contiguous, 32B aligned) │ │ - magic │ │ Vec[0]: [f32; D] │ │ - version │ │ Vec[1]: [f32; D] │ │ - count │ │ … │ │ - dim │ │ Vec[N-1]: [f32; D] │ │ - checksum │ │ │ └─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐ │ Neighbor Block (64KB aligned) │ ├─────────────────────────────────────────────────────────────────┤

Design Principles

1. SIMD Alignment: All vectors 32-byte aligned for AVX2
2. Cache Lines: Hot data packed into 64-byte cache lines
3. Prefetching: Neighbor lookups prefetch next block
4. Contiguous: Avoid pointer chasing in hot path

Structs

BatchDistanceComputer

Batch distance computation with prefetching

EmbeddingBlockHeader

Header for an embedding block

EmbeddingStorage

Contiguous, SIMD-aligned embedding storage

HotPathVectorStore

Combined embedding and neighbor storage optimized for HNSW traversal

NeighborBlockHeader

Header for a neighbor block

NeighborStorage

Contiguous neighbor list storage for graph traversal

Constants

BLOCK_ALIGNMENT

Block alignment (page size)

CACHE_LINE_SIZE

Cache line size

EMBEDDING_MAGIC

Magic number for embedding blocks

NEIGHBOR_MAGIC

Magic number for neighbor blocks

SIMD_ALIGNMENT

SIMD alignment for vectors (AVX2 = 32 bytes, AVX-512 = 64 bytes)

Functions

align_down

Align a value down to the given alignment

align_up

Align a value up to the given alignment

alloc_aligned

Allocate aligned memory

free_aligned^⚠

Free aligned memory