Expand description
Backfill job: adds a new vector column to existing files in an AI-Lake table.
Reads each file, generates embeddings for the new column via embed_fn, and
rewrites the file with both the original vector column and the new one (using
write_multi). Commits an Overwrite snapshot per file (AtomicReplace semantics).
Idempotent: files that already contain the new column (detected via
has_column_footer) are skipped.
Re-exports§
pub use crate::migration::EmbedFn;
Structs§
- Backfill
Job - Adds a new vector column to all existing files in a table.
- Backfill
Progress - Progress reported after each file is backfilled.