Struct ninja_build::deps_log::DepsLog

pub struct DepsLog {}

As build commands run they can output extra dependency information (e.g. header dependencies for C source) dynamically. DepsLog collects that information at build time and uses it for subsequent builds.

The on-disk format is based on two primary design constraints: - it must be written to as a stream (during the build, which may be interrupted); - it can be read all at once on startup. (Alternative designs, where it contains indexing information, were considered and discarded as too complicated to implement; if the file is small than reading it fully on startup is acceptable.) Here are some stats from the Windows Chrome dependency files, to help guide the design space. The total text in the files sums to 90mb so some compression is warranted to keep load-time fast. There's about 10k files worth of dependencies that reference about 40k total paths totalling 2mb of unique strings.

Based on these stats, here's the current design. The file is structured as version header followed by a sequence of records. Each record is either a path string or a dependency list. Numbering the path strings in file order gives them dense integer ids. A dependency list maps an output id to a list of input ids.

Concretely, a record is: four bytes record length, high bit indicates record type (but max record sizes are capped at 512kB) path records contain the string name of the path, followed by up to 3 padding bytes to align on 4 byte boundaries, followed by the one's complement of the expected index of the record (to detect concurrent writes of multiple ninja processes to the log). dependency records are an array of 4-byte integers output path id, output path mtime, input path id, input path id... If two records reference the same output the latter one in the file wins, allowing updates to just be appended to the file. A separate repacking step can run occasionally to remove dead records.

Methods

impl DepsLog

fn new() -> Self

fn load(
    &mut self,
    path: &Path,
    state: &mut State
) -> Result<Option<String>, String>

fn open_for_write(&self, path: &Path) -> Result<(), String>

fn record_deps(
    &self,
    state: &State,
    node_idx: NodeIndex,
    mtime: TimeStamp,
    nodes: &[NodeIndex]
) -> Result<(), ()>

fn recompact(&self, path: &Path) -> Result<(), String>