Module feature_flags 

Feature flags

Constants

ENABLE_UNSTABLE_DATA_OVERLAY_FILES_ENV

Environment variable that opts a release build into reading and writing data overlay files before the feature is generally released.

FLAG_BASE_PATHS

Dataset uses multiple base paths (for shallow clones or multi-base datasets)

FLAG_COVERED_INDEX_METADATA

Some index declares covering columns: IndexMetadata.covering_fields names columns the index carries values for, whether or not it is also keyed on them.

FLAG_DELETION_FILES

Fragments may contain deletion files, which record the tombstones of soft-deleted rows.

FLAG_DISABLE_TRANSACTION_FILE

Disable writing transaction file under _transaction/, this flag is set when we only want to write inline transaction in manifest

FLAG_FRAGMENT_REUSE_INDEX

Tagged FRI requires a reader that interprets its mappings and a writer that preserves them during maintenance. Legacy-only FRI does not set this bit. Bit 9 is taken by the stable-row-id FRI compatibility flag.

FLAG_FRAGMENT_TREE

Reserved for fragment trees. Readers and writers reject it until implemented.

FLAG_FRAG_REUSE_WITH_STABLE_ROW_IDS

The table uses stable row ids and carries a fragment reuse index.

FLAG_INDEPENDENT_COVERING_FIELDS

Index key fields and covering fields are declared independently.

FLAG_MANAGED_BLOBS

Blob v2 descriptors may independently address Lance-owned objects. Readers must resolve their explicit bases and writers/GC must preserve those references. This capability is sticky, including across restore, and requires both words.

FLAG_MIXED_DATA_FILE_VERSIONS

A dataset may reference recognized V2 data files with different exact versions. Readers and writers must both understand the per-file version contract before either can safely access the dataset.

FLAG_NON_REUSABLE_FIELD_IDS

Field IDs are allocated from a persistent high-water mark and are never reused. Writers must understand this allocation contract. It does not change how readers interpret the schema or data files.

FLAG_OBJECT_METADATA

Writer-only protection for clustering definitions, ID allocation, references, and generic fragment/index segment metadata maintenance. Reserved and unsupported until writers implement its maintenance rules. Ordinary readers can ignore this metadata; all covered features share this one bit.

FLAG_SPILLED_ROW_LINEAGE

Some fragment stores a row lineage sequence – its row ids, or its created-at or last-updated-at versions – as a hidden column of a data file rather than inline in the manifest (RowIdMeta::Column, RowDatasetVersionMeta::Column).

FLAG_STABLE_ROW_IDS

Row ids are stable for both moves and updates. Fragments contain an index mapping row ids to row addresses.

FLAG_TABLE_CONFIG

Table config is present

FLAG_UNKNOWN

The first bit that is unknown as a feature flag

FLAG_UNSTABLE_DATA_OVERLAY_FILES

Fragments contain data overlay files, which supply new values for a subset of cells without rewriting base data files. A reader that does not understand overlays must refuse the dataset, since ignoring an overlay would silently return stale base values.

FLAG_UNSTABLE_SPILLED_ROW_LINEAGEDeprecated

Deprecated name for FLAG_SPILLED_ROW_LINEAGE.

FLAG_USE_V2_FORMAT_DEPRECATED

Files are written with the new v2 format (this flag is no longer used)

FLAG_ZONE_WRITE_SEEDS

Required for data files containing version 1 zone-level statistics seeds. Their min/max exclude NaNs, which are counted separately by sign. Set in both reader and writer feature flags and preserve across commits.

MANAGED_BLOBS_META_KEY

Schema metadata that explicitly opts writes and compaction into Managed blobs. Set lance.blob.managed to true to activate the sticky table capability. This requires Managed-aware readers and maintenance clients thereafter.

Functions

apply_feature_flags

Set the reader and writer feature flags in the manifest based on the contents of the manifest.

can_read_dataset

can_write_dataset

ensure_can_read_manifest

Refuse reads from manifests whose required reader features this build does not support or whose paired capabilities are inconsistent.

ensure_can_write_manifest

Refuse writes to manifests whose required writer features this build does not support or whose paired capabilities are inconsistent.

has_deprecated_v2_feature_flag

inherit_sticky_feature_flags

Carry sticky capabilities from the manifest a new one is derived from.

managed_blobs_enabled

Whether a schema explicitly requests Managed blob storage. Absent or false keeps the existing Inline/Packed/Dedicated storage layout, in all build profiles.

validate_non_reusable_field_id_flags

Refuse a manifest whose non-reusable-field-ID marker and required flags disagree.

validate_paired_feature_flags

Refuse a manifest whose paired reader and writer capability bits disagree.