Module local_copy 

A local copy of a remote dataset’s objects, for Data Quality’s full scan.

A full scan is several passes over its scope, and over an object store each pass reads the objects again. Within a byte budget the run fetches each object once into the cache directory and makes its passes over the copy instead. The copy is a session snapshot: it is kept for later full scans of the same dataset until it is released, the dataset is opened again or replaced, or datui exits. A copy that did not finish is removed when the fetch stops.

Structs

LocalCopy

Objects copied from a remote dataset, each at the path its URL maps to. Removed from disk when the last holder drops it: the app’s retained copies and any run reading it.

RemoteObject

One remote object a copy fetches, as the open found it.

Constants

COPIES_DIR

Where copies live under the cache directory.

Functions

free_space

Free bytes on the file system holding dir, or its nearest existing parent.

same_etag

Whether the tag a fetch was answered with is the one listed: stores quote it in one answer and not the other, and mark a weak one.

sweep

Remove copies under root no session holds: left by a datui that did not exit cleanly, or quit while a run read one. A copy another running datui holds is locked and stays, and so does one too new to judge.