Struct CompactionPlanner 

pub struct CompactionPlanner { /* private fields */ }

Implementations

impl CompactionPlanner

pub fn new(config: CompactionConfig) -> Self

pub fn plan(&self, files: &[DataFileEntry]) -> Vec<DataFileEntry>

Select files to compact.

Picks files smaller than target_file_size_bytes, sorts them smallest-first (cheapest to read), and caps the selection at max_files_per_pass. This tiered approach prevents a single pass from compacting the entire table into memory when thousands of small files exist.

Foreign-write detection: every file written by the AI-Lake SDK always carries a centroid_b64 (see make_data_file_entry/make_data_file_entry_indexing — centroid is computed and stored even for IndexStatus::Indexing files, before the HNSW build itself completes). A manifest entry with no centroid was never produced by AI-Lake — almost certainly a generic Iceberg engine (Spark/Trino OPTIMIZE / rewrite_data_files, DuckDB) rewrote the file with no knowledge of AI-Lake. Every query against it silently degrades to an O(N) flat scan (see the flat_scan_unexpected warning in scanner.rs::search). These files bypass both the size filter and min_files_to_compact — a single such file is worth repairing on its own; it shouldn’t wait for enough small files to accumulate.