Trait Extractor

Source

pub trait Extractor {
    // Required method
    fn extract(&self, path: &str, blob_id: &str, bytes: &[u8]) -> FactSet;

    // Provided methods
    fn env_tag(&self) -> u64 { ... }
    fn reads(&self, path: &str) -> bool { ... }
    fn mines(&self, path: &str) -> bool { ... }
    fn paths(&self) -> &PathPolicy { ... }
}

Expand description

Turns one source blob into the nodes and edges derived from it.

Required Methods§

Source

fn extract(&self, path: &str, blob_id: &str, bytes: &[u8]) -> FactSet

Extract a FactSet from a blob’s path, git blob_id, and bytes.

Implementations must be deterministic: identical inputs must always produce an identical fact set.

Provided Methods§

Source

fn env_tag(&self) -> u64

Runtime inputs — beyond (path, bytes) — that change extraction output and so must be folded into the sync cache key: the installed OCR-model identity and any IngestConfig toggles that gate extraction. The default is the media-model tag alone; Registry additionally folds in its ingestion config so toggling content off re-extracts affected blobs instead of serving stale, content-bearing facts.

Source

fn reads(&self, path: &str) -> bool

Whether sync should read this path’s bytes at all.

false for a PathClass::Excluded path, so an excluded corpus is not pulled out of the object database only to be discarded — which for the raw/ case ADR-0026 step 1 describes is gigabytes of PDFs, and is also the difference between “the scan does not read this” and “the scan reads this and throws it away”.

It is an optimisation with a safe failure direction, not a second copy of the rule: an implementation that wrongly returns true still gets an empty fact set out of Extractor::extract, because both answers come from the one PathPolicy. The default admits everything, so an implementation that ignores it behaves exactly as before.

Source

fn mines(&self, path: &str) -> bool

Whether anything may be derived from what this path holds, beyond a file’s identity.

Separate from Extractor::reads because the two answer different questions and an crate::PathClass::Opaque path answers them differently: its bytes are read (its length is a fact about it) while nothing may be mined from what they say. sync asks this for the facts it assembles outside Extractor::extract — the submodule nodes it appends after flattening, which come from .gitmodules rather than from any one blob and so never pass through the extractor at all.

The default admits everything, so an implementation that ignores it behaves exactly as before.

Source

fn paths(&self) -> &PathPolicy

The policy itself, for the one reader that needs to ask about a path it has not been handed.

Extractor::reads and Extractor::mines answer about a given path, which suits every reader that is iterating one. The submodule reader is different: it reads .gitmodules to derive facts about vendor/dep, so the decision about the source file happens inside rto_graph::git, below the extractor, and that code needs the policy rather than an answer about a path chosen for it.

Defaults to the empty policy, so an implementation that ignores it behaves exactly as before.