Module rules 

The document-format index manifest: what an app declares it wants indexed.

The manifest is stored as JSON in doc_formats.search and parsed into IndexRules here. It names RTDB collections (kind) and the fields within their documents that carry text, so indexing a new app needs no Rust.

A worked example — notillo, whose pages live in collection p and whose blocks live in b and point at their page through field p:

{
  "v": 1,
  "parts": [
    { "kind": "p", "title": ["ti"], "tags": ["tg"], "parent": "pp" },
    { "kind": "b", "attachTo": { "kind": "p", "field": "p" },
      "anchor": "docId", "order": ["o"],
      "prune": ["$..c[0:][1:]", "$..cells[0:][0:][1:]"],
      "body": [
        { "path": "c", "extract": "text", "keys": ["c", "cells", "wt"] },
        { "path": "$..tg", "extract": "string", "prefix": "#" },
        { "path": "pr.caption" }
      ] }
  ],
  "limits": { "maxParts": 5000, "maxBodyChars": 100000 }
}

A part without attachTo emits one index row per document. A part with attachTo emits nothing of its own — its text is folded into the body of the owning part’s row. That is what makes the index deep: block text lands on the page row, so a hit deep-links to the page.

Field rules

A field rule’s path (spelled field in manifests written before the rename, which is still accepted) is either a dotted path ("pr.caption", numeric segments indexing arrays) or an RFC 9535 JSONPath query, told apart by the leading $ the standard requires. extract then says how the selected nodes become text: "text" (the default) walks the node for string leaves, "string" takes the node verbatim and skips it if it is not a string.

Four modifiers shape what a walk emits. keys is an allowlist of object keys whose strings are prose, excludeKeys a denylist of keys whose subtree is skipped, prefixKeys prefixes the strings under a named key, and prefix prefixes every token the rule emits. The modifiers compose, so a manifest can spend one rule per text source rather than making one rule do everything — but note that crate::extract::extract_fields appends every rule’s output into one sink in declaration order, so a single prose stream must stay a single rule. Splitting one would scramble reading order; only metadata (tags, captions) belongs in a rule of its own, trailing the text it annotates.

Pruning

A part rule’s prune is a list of RFC 9535 queries whose matches are deleted from the document before any field rule sees it. It exists for the one thing keys structurally cannot do: keys gates by name, prune gates by position. notillo stores a styled run as the positional tuple ["szöveg", "b"], so the style flag shares its text’s enclosing key and no allowlist can separate them — but $..c[0:][1:] names the tail slot directly.

Deletion-only is what keeps it safe. A prune pattern can remove text; it can never reorder or fabricate any, so the single-ordered-walk invariant the field rules rest on survives untouched. A pattern that fails at run time therefore degrades to “the flag tokens stay in the index”, which is the status quo.

Prefer the slice [0:] to the wildcard [*]. A slice is inert on anything that is not an array (process_slice ends in as_array().map(…).unwrap_or_default()); a wildcard descends objects too. notillo’s table block stores its content as the object {"type": "tableContent", "cw": […], "rows": [{"cells": […]}]} under the same c key an ordinary block uses for its inline array. $..c[0:][1:] sees nothing there and leaves the table alone, whereas $..c[*][1:] would descend into that object, reach rows, and delete every row but the first.

See crate::prune for the evaluation order and the error handling.

The same FieldRule vocabulary is reused for actions, whose manifests come from the Action DSL rather than from doc_formats — see ActionSearchRules.

Structs

ActionSearchRules

An action type’s search manifest, declared in the Action DSL’s ActionDefinition::search and parsed here.

AttachTo

Where an attached part’s text goes.

FieldRule

One text source within a document.

IndexRules

Parsed and validated index manifest.

Limits

Guard rails. Exceeding any of these truncates and warns; it never fails the index run, because a partially indexed document beats an unindexed one.

PartRule

One collection’s indexing rule.

Enums

ExtractMode

How the text of a selected node is taken.

Selector

What a field rule’s field string selects out of a document.

Constants

DOC_ID

Sentinel anchor / order value meaning “the document’s own RTDB id” rather than a field inside it. export_all returns ids as keys, not as a field, so there is nothing else to name them by.

SUPPORTED_VERSION

Manifest version this build understands. A higher v is refused rather than half-applied.