Skip to main content

Module prune

Module prune 

Source
Expand description

Deleting manifest-named nodes from a document before extraction.

A part rule’s prune list (crate::rules::PartRule::prune) names nodes that are not text. They are deleted from the exported document before crate::extract walks it, so a rule’s keys allowlist only ever sees prose.

§Why by position and not by name

keys gates a leaf by its enclosing object key, which cannot separate the members of a positional tuple: notillo stores a styled run as ["szöveg", "b"], where slot 1 is a style flag drawn from the closed vocabulary b i u s c. Every such flag is a subsequence of "biusc", so "is" and "bus" — real words — become index tokens and cause false hits. Both slots share one enclosing key, so no allowlist can tell them apart. A JSONPath selector could name slot 0, but crate::extract::extract_fields appends every rule’s output into one sink in declaration order, so splitting one prose stream across several rules scrambles reading order. Deleting the tail slot up front leaves one rule, one walk, one reading order.

§Why deletion-only is safe

Pruning can only ever remove text: it loses no text it was not aimed at and fabricates nothing, so a pattern that fails at run time degrades to “a few style-flag tokens survive”. That is why a failure here is a warn! rather than an error: propagating would abort a scheduler task that retries on a timer forever, for a manifest problem that only degrades the index.

Object deletion does not preserve order, though: under serde_json/preserve_order Map::remove is a swap-remove, which is what jsonpath-rust calls, so the object’s last key lands in the freed slot and its siblings’ extraction order shifts. Array-element deletion keeps order — one more reason to prefer the slice patterns below.

§Slices, not wildcards

Write [0:], not [*]. jsonpath-rust’s slice selector is inert on a non-array while its wildcard descends objects as well; see crate::rules for the notillo table shape that makes the difference bite.

§Cost

Each pattern is a whole extra traversal of the document, plus one allocated normalised path per match and one reparse of that path inside delete_by_path. crate::rules’s MAX_PRUNE_RULES is the only bound: MAX_JSONPATH_NODES does not apply here, because a deletion’s match set is built inside delete_by_path where there is no hook to cap it. Peak extra memory is one short String per match, against a document already fully resident as a Value.

Functions§

prune_docs
Prune every exported document against the rules for its kind.
prune_document
Apply one part rule’s prune list to one document, in declaration order.