Struct LazyFlatVectorData 

pub struct LazyFlatVectorData {
    pub dim: usize,
    pub num_vectors: usize,
    pub quantization: DenseVectorQuantization,
    pub doc_ids: Vec<(u32, u16)>,
    /* private fields */
}

Lazy flat vector data — doc_ids in memory, vectors accessed via range reads.

Only the doc_id index (~6 bytes/vector) is loaded into memory. Vector data stays on disk and is accessed via mmap-backed range reads. Element size depends on quantization: f32=4, f16=2, uint8=1 bytes/dim.

Used for:

• Brute-force search (batched scoring with native-precision SIMD)
• Reranking (read individual vectors by doc_id via binary search)
• doc() hydration (dequantize to f32 for stored documents)
• Merge streaming (chunked raw vector bytes + doc_id iteration)

Fields

dim: usize

Vector dimension

num_vectors: usize

Total number of vectors

quantization: DenseVectorQuantization

Storage quantization type

doc_ids: Vec<(u32, u16)>

In-memory doc_id index: (doc_id, ordinal) per vector

Implementations

impl LazyFlatVectorData

pub async fn open(handle: LazyFileSlice) -> Result<Self>

Open from a lazy file slice pointing to the flat binary data region.

Reads header (16 bytes) + doc_ids (~6 bytes/vector) into memory. Vector data stays lazy on disk.

pub async fn read_vector_into(&self, idx: usize, out: &mut [f32]) -> Result<()>

Read a single vector by index, dequantized to f32.

out must have length >= self.dim. Returns Ok(()) on success. Used for ANN training and doc() hydration where f32 is needed.

pub async fn get_vector(&self, idx: usize) -> Result<Vec<f32>>

Read a single vector by index, dequantized to f32 (allocates a new Vec).

pub async fn read_vector_raw_into( &self, idx: usize, out: &mut [u8], ) -> Result<()>

Read a single vector’s raw bytes (no dequantization) into a caller-provided buffer.

out must have length >= self.vector_byte_size(). Used for native-precision reranking where raw quantized bytes are scored directly.

pub async fn read_vectors_batch( &self, start_idx: usize, count: usize, ) -> Result<OwnedBytes>

Read a contiguous batch of raw quantized bytes by index range.

Returns raw bytes for vectors [start_idx..start_idx+count). Bytes are in native quantized format — pass to batch_cosine_scores_f16/u8 or batch_cosine_scores (for f32) for scoring.

pub fn flat_indexes_for_doc(&self, doc_id: u32) -> (usize, &[(u32, u16)])

Find flat indexes for a given doc_id via binary search on sorted doc_ids.

doc_ids are sorted by (doc_id, ordinal) — segment builder adds docs sequentially. Returns a slice of (doc_id, ordinal) entries; the position of each entry in self.doc_ids is its flat vector index.

Returns (start_index, slice) where start_index is the position in doc_ids.

pub fn get_doc_id(&self, idx: usize) -> (u32, u16)

Get doc_id and ordinal at index (from in-memory index).

pub fn vector_byte_size(&self) -> usize

Bytes per vector in storage.

pub fn vector_bytes_len(&self) -> u64

Total byte length of raw vector data (for chunked merger streaming).

pub fn vectors_byte_offset(&self) -> u64

Byte offset where vector data starts (for direct handle access in merger).

pub fn handle(&self) -> &LazyFileSlice

Access the underlying lazy file handle (for chunked byte-range reads in merger).

pub fn estimated_memory_bytes(&self) -> usize

Estimated memory usage (only doc_ids are in memory).

Trait Implementations

impl Clone for LazyFlatVectorData

fn clone(&self) -> LazyFlatVectorData

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LazyFlatVectorData

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.