Skip to main content

select_load_keys

Function select_load_keys 

Source
pub fn select_load_keys(
    new: &[(String, RunManifest)],
    all_parquet: &[String],
) -> Vec<String>
Expand description

Pure selection: which bucket-relative parquet keys to load for the given (not-yet-loaded) run manifests.

Prefers each manifest’s own parts (via [resolve_parts]) intersected with all_parquet — so a load pulls exactly the new runs’ files, not every object under the prefix (the key to incremental loads once cleanup_source no longer wipes the bucket). Falls back to the whole all_parquet listing when ANY new manifest resolves to no present part (legacy/part-less manifests still load, at the cost of not pruning); the row-count gate then still guards correctness.