Module checkpoints

Implementation for writing delta checkpoints.

Constants

CHECKPOINT_RECORD_BATCH_SIZE

The record batch size for checkpoint parquet file

Functions

cleanup_expired_logs_for

Deletes all delta log commits that are older than the cutoff time and less than the specified version.

cleanup_metadata

Delete expires log files before given version from table. The table log retention is based on the logRetentionDuration property of the Delta Table, 30 days by default.

create_checkpoint

Creates checkpoint at current table version

create_checkpoint_for

Creates checkpoint for a given table version, table state and object store

create_checkpoint_from_table_uri_and_cleanup

Loads table from given table_uri at given version and creates checkpoint for it. The cleanup param decides whether to run metadata cleanup of obsolete logs. If it’s empty then the table’s enableExpiredLogCleanup is used.