Struct TextModel 

pub struct TextModel { /* private fields */ }

A Quarb adapter over a text-level document.

Implementations

impl TextModel

pub fn tokenize_given(&mut self, sentences: &str) -> Result<(), String>

A .sentences file’s lines, aligned to the text: one line per sentence in reading order (blank lines ignored, headings as lines), each matched whitespace-insensitively at the current position; the first line that does not match is refused by its number naming the text found, and the first block the file leaves uncovered by its path. Line k is then the sentence with ::::n k.

impl TextModel

pub fn build(blocks: Vec<Block>) -> Self

Assemble the document tree from a adapter’s event stream.

Iterative throughout (the stream is flat; prose flattening runs over indices), so pathological nesting cannot overflow the call stack. Lenient on malformed streams: a stray Close is ignored, unclosed containers close at the end.

pub fn set_bib_aliases(&mut self, map: HashMap<String, String>)

Register the bib-field alias census (alias → canonical campus), lowercased keys — the friction remover: a field authored or queried under any covered name answers.

pub fn set_document_url(&mut self, url: &str)

Declare the document’s own URL — the base a relative ref target joins against when --> reaches across documents.

pub fn set_link_base(&mut self, url: &str)

Declare the base relative reference targets join against — html’s <base href> — when it differs from the document’s own URL.

pub fn set_document_path(&mut self, path: &str)

Declare where the document is served from when it declares no absolute URL of its own: the base ::href links against (a mount path, an archive member’s path).

pub fn omit_notes(&mut self, omit: bool)

The scheme ::cite reads: the mount’s ?scheme=. Leave the note bodies out of the corpus reading’s tier (?notes=omit); set before tokenizing.

pub fn lower_references(&mut self, standard: &str) -> Result<usize, String>

Lower the reference list a document carries as prose (?references=STANDARD): every paragraph that reads as a reference under the standard becomes a bib node — its text the projection, the fields under the canonical campi, the genus as trait and ::::genus, ::onym the list’s number or else the responsible party and year — and the citations in the remaining prose become cit nodes (bracketed numbers and ranges with their pinpoint, author- year parentheses) resolved against them. A paragraph that does not parse stays prose; no heading is matched. Returns the number of entries lowered; an unknown standard is an error naming the known ones.

pub fn set_cite_scheme(&mut self, scheme: &str)

pub fn set_orthography(&mut self, table: &str) -> Result<(), String>

The spelling table the tokens fold through: the mount’s ?modernize=. An unknown table is an error.

pub fn set_head_meta(&mut self, head: HeadMeta)

Attach what the document declares about itself; the format adapter reads it from the source’s own metadata vocabulary.

pub fn head_meta(&self) -> &HeadMeta

What the document declares about itself.

pub fn plain_text(&self) -> String

The document’s text as one string: every section’s lemma and every block’s prose, one per line, in document order — the lowering a substrate stores beside the document so a substring predicate inside the graft can be prefiltered on it. The invariant a prefilter relies on: every grafted node’s prose (and every lemma) is a substring of this.

pub fn node_count(&self) -> usize

How many nodes the document holds (the root included) — the bound a host needs to pack this document’s node ids into a wider id space.

pub fn parse_plain(text: &str) -> Self

Read plain text: blank-line-separated paragraphs, each collapsed to one line — the atramento paragraph rule. No headings, no markup.

pub fn locator(&self, node: NodeId) -> String

A locator path to node, like /section[2]/paragraph[3], for rendering. A [n] index is added only to disambiguate same-name siblings.

impl TextModel

pub fn apply_cast(&mut self, rows: &[CastRow])

The corpus reading: give every prose block its tokens as children. Segmentation is UAX #29 word boundaries (the standard sentences already follows): don’t, Tom’s and 3,000 are one token each, an em dash is its own, and whitespace is none. Positions and sentence ordinals count through the document in reading order. The block’s own :: is unchanged; the tokens sit after any children it already had (an inline ref, a callout). The literary reading’s cast layer: declare the characters a cast table names (those the source did not), annotate every word token a row’s pattern matches with the row’s id as its prosopon, then resolve every mention to its character.

pub fn apply_places(&mut self, rows: &[CastRow])

The literary reading’s gazetteer: a places table declares the places a source names, as a cast table declares its people. Each row is a place node under the root (::lemma its name, ::onym its id, every other column a field — ::lat, ::lon, ::kind); a word token a row’s pattern matches answers the row’s id as ::chora and reaches the place as ->place, and the place reaches its mentions back as <-place.

pub fn tokenize(&mut self)

pub fn corpus_text(&self) -> String

The document’s prose as one text — every prose block’s ::, blank-line separated, in reading order — the text an external annotator reads and a CoNLL-U source aligns to.

pub fn is_tokenized(&self) -> bool

Whether the corpus reading’s tokens are already in place.

pub fn tokenize_with(&mut self, bonds: Option<&Syndesmos>)

tokenize with sentence bonds: a .desm set (the syndesmos crate) whose abbreviations and patterns undo UAX #29 breaks.

pub fn annotate_conllu(&mut self, conllu: &str) -> Result<(), String>

The corpus reading from a CoNLL-U annotation (the corpus reading’s second layer, carrying every column): the source’s sentences and tokens replace the built-in segmentation wholesale, aligned to the document’s prose by character offset — each token’s form is found at the cursor, whitespace skipped — and every token carries the source’s lemma, parts of speech, features, dependency relation, head edge, enhanced heads and miscellany. A form that does not align is an error naming the offset: the annotation must be of this text. A multiword range (1-2 don't) aligns as the unit and its syntactic words share its span; an empty node (8.1) is not in the text and consumes none of it.

pub fn parse_conllu_text(text: &str) -> Result<Self, String>

A treebank read as a document: the file is the root; # newdoc opens a section, # newpar a paragraph, each sentence block a sentence node whose :: is its # text (else the forms joined per SpaceAfter) and whose comments answer as properties. No tokens: the reading a bare .conllu or text: takes.

pub fn parse_conllu_corpus(text: &str) -> Result<Self, String>

A treebank read as a corpus: parse_conllu_text’s document with the file’s own tokens as each sentence’s children, in id order, every column carried — the reading corpus: takes on a .conllu.

pub fn parse_conllu_text_set(files: &[(&str, &str)]) -> Result<Self, String>

A set of treebank files read as one document — a UD treebank’s train / dev / test files, a release directory: each file a level-1 section under the root whose ::lemma is the name given, the file’s own # newdoc sections nested beneath it. No tokens, as parse_conllu_text.

pub fn parse_conllu_corpus_set(files: &[(&str, &str)]) -> Result<Self, String>

parse_conllu_text_set with every file’s tokens under its sentences: token positions, sentence ordinals and mention ordinals run across the whole set, as they do across one file.

Trait Implementations

impl AstAdapter for TextModel

fn estimate(&self, scope: &[&str], plan: &StepPlan<'_>) -> Option<Estimate>

The shelf knows how many documents, sections and tokens a step reaches; nothing is read again.

fn walks_for_references(&self) -> bool

In memory: an unscoped reverse resolution or value match may walk the arbor.

fn descendants_where( &self, node: NodeId, name: &str, property: &str, value: &Value, ) -> Option<Vec<(NodeId, usize)>>

The positional index: //token[::lower = "word"] below any node answers from the form → tokens map, scoped to the node’s subtree. Only a plain word is answered — a literal with a numeric, temporal, durational or unital reading is left to the walk, whose equality reads it.

fn traits(&self, node: NodeId) -> Vec<String>

The <block> family on every block node, plus <table> on a list that denormalizes a table. Kinds are node names, not traits.

fn property(&self, node: NodeId, name: &str) -> Option<Value>

::lemma (title), ::hypograph (footer or attribution), ::taxis (ordinal), ::text (the flattened prose, same as the bare projection). The Greek anatomy — ::lemma, ::grammata, ::hypograph, ::taxis — plus the friendly aliases (::title, ::body, ::attribution, ::ord), answered here because this adapter’s property surface IS the vocabulary; on data adapters those spellings stay ordinary field names. The Greek is canon in docs and reflection preserves whichever spelling was written.

fn default_value(&self, node: NodeId) -> Option<Value>

The default projection is the flattened prose of the subtree — lemma first, hypograph last.

fn links(&self, node: NodeId) -> Vec<(String, NodeId)>

The edge label is the family name both ends share — ->footnote / ->endnote — the node’s own kind, since resolution re-kinds an open-family callout to its body.

fn link_property( &self, source: NodeId, label: &str, target: NodeId, name: &str, ) -> Option<Value>

$-::rel on a dependency edge: the relation — ->head’s is the dependent’s ::deprel, ->ehead’s the enhanced relation to that head.

fn resolve( &self, node: NodeId, property: &str, hint: Option<&str>, ) -> Option<NodeId>

//ref-->: land on the bearer — the block if a block bears the name, the point anchor if a point does. The text-level hints choose the reading: -->block normalizes a point landing to its enclosing block, -->point selects point targets only; bare is the exact landing.

fn ref_property(&self, node: NodeId) -> Option<String>

The bare arrow’s property: a ref resolves its target.

fn external_ref( &self, node: NodeId, property: &str, hint: Option<&str>, ) -> Option<ExternalRef>

A ref the producer declared external: the absolute URL, relative targets joined against the document’s own URL — the same acquisition rung the html DOM reading speaks.

fn resolve_fragment(&self, _node: NodeId, fragment: &str) -> Option<NodeId>

A sibling document’s #fragment lands here: the bearer of that name — a labeled block, or a point anchor.

fn aliased_metadata(&self, _node: NodeId) -> &'static [&'static str]

Ruling #29: the text level’s surface is the vocabulary itself — no document can introduce a property name — so its two annotations answer at :: as well.

fn metadata(&self, node: NodeId, key: &str) -> Option<Value>

::::level on sections (the source heading level) and ::::lang on verbatim blocks (the declared language).

fn root(&self) -> NodeId

The root node — the initial navigation context.

fn children(&self, node: NodeId) -> Vec<NodeId>

The tree children of node, in document order.

fn name(&self, node: NodeId) -> Option<String>

The name of node — the label of its incoming tree edge.

fn parent(&self, node: NodeId) -> Option<NodeId>

The parent of node, or None for the root.

fn backlinks(&self, node: NodeId) -> Vec<(String, NodeId)>

Incoming crosslinks to node, as (label, source) pairs, for <- navigation. May be expensive (an adapter that does not precompute edges must search for referrers).

fn properties(&self, _node: NodeId) -> Vec<(String, Value)>

The node’s own properties, enumerated — name and value, in the source’s order — for readings that keep values off the tree (the JSON property reading, where an object’s scalar fields are no nodes). The serializers (| json, | xml, | kaiv) merge them with the children, a child winning a shared name. The default is none: for a reading whose values are all nodes, the children already say everything.

fn can_refer_to(&self, source: NodeId, _target: NodeId) -> Option<bool>

Whether a reference declared on source — a catalog key, a model ref, a reference property the format defines — can land in target’s container. A question of the schema, never of the data; None where nothing is declared at the node. The default: a node with a reference property of its own can refer anywhere (an href may point at any element).

fn reference_properties(&self, source: NodeId, _target: NodeId) -> Vec<String>

The reference properties declared on source that can land in target’s container: a catalog’s key columns into the target’s table, the fields of a model’s refs, the node’s own reference property. What t<-- and t<--(SCOPE) read when the query names no property; empty where none is declared.

fn resolve_needs_label(&self, _node: NodeId, _property: &str) -> bool

Whether resolving property of node needs a reference label, because the property itself declares no target: a column with no key, or any property of a property graph, where a value names a node only under a node label. A label-less ::p--> over such a property is refused with the remedy, since an empty answer would read as “points nowhere”.

fn reverse_label(&self, _target: NodeId) -> Option<String>

The reference label under which a referrer’s property is also resolved when a reverse arrow asks about target and the query writes no label. None (the default): only the label-less resolution counts. An adapter whose label names a container (a table, a collection, a node label) returns the one target stands in, so /authors/1<--(/books/*::author_id) reads author_id as a key into authors without the query saying so. The engine derives no label of its own.

fn tree_identities(&self) -> Vec<(String, NodeId)>

The trees of this arbor other trees refer to by name: each identity (a URL, a path, any string a reference may hold) with the node that stands for the tree. An identity ending in a separator is a base: every identity beneath it names the node reached beneath the base’s node by the remainder’s segments. Declared in a model (tree IDENTITY: PATH;) and automatic for a mount’s own target; the engine reads nothing inside an identity beyond prefix and separator.

fn property_names(&self, node: NodeId) -> Vec<String>

The names of the properties a node has — what “any property” means for <--(SCOPE::*). By default its off-tree properties, its reference property, and its children’s names (a row’s columns, an object’s keys are properties under the same names); an adapter whose properties are not children (an element’s attributes) lists them.

fn resolves_references(&self) -> bool

Whether this source resolves references at all. An adapter with no notion of a reference (a CSV file, a directory, a log stream) says no, and a reverse arrow over it is refused with the value match as the remedy, rather than answered with an empty set that would read as “nothing points here”.

fn index_lookup( &self, _scope: &str, _property: &str, _value: &Value, ) -> Option<Vec<NodeId>>

index SCOPE::prop; — the nodes of the declared scope whose property holds value, from an index the model keeps; None when no declaration covers the pair (the walk answers). An index changes cost, never the answer: the caller still tests each node it names.

fn index_referrers( &self, _scope: &str, _property: &str, _label: Option<&str>, _target: NodeId, ) -> Option<Vec<NodeId>>

The declared-scope nodes whose property resolves (under the relation label, when given) to target, from the same index; None when no declaration covers the pair.

fn non_tree_nodes(&self) -> Vec<NodeId>

The nodes of the arbor that are not in the tree — joined to it by crosslinks alone, with no parent and no path. The universal search *// reaches them after the tree, in this order; :::is-non-tree is true on them. The default is none: every shipped adapter puts every node in the tree.

fn child_host(&self, node: NodeId) -> NodeId

The node whose children an explicit child step from node reads. The default is node itself. An adapter whose tree holds transparent links (a filesystem’s symlink to a directory) answers the link’s target, so /link/name lands on the target’s own child — the same node /target/name reaches. The descendant walk never consults this: it reads children alone, so a linked subtree is walked once, under its own parent, and a cycle of links cannot trap it.

fn children_named(&self, node: NodeId, name: &str) -> Vec<NodeId>

The children of node whose edge name is exactly name — the engine’s fast path for name-matcher child hops. The default filters children; an adapter whose containers cannot be enumerated (permission-scoped or unbounded remote trees) overrides this with a direct, name-addressed lookup. Must be observationally identical to the default wherever enumeration works. The adapter owns the name test: it may deliberately alias — resolve a name to a node whose edge name differs (git revision syntax landing on a hash-named commit) — and the engine will not re-filter. Container-scoped resolution like that stays child-axis only; per-node spelling aliases belong in answers_to, which this default consults.

fn answers_to(&self, node: NodeId, name: &str) -> bool

Whether node answers to name as a literal spelling . The default is canonical equality. An adapter may declare per-node aliases — a social feed spelling a handle @alice while the stripped alice is the hop name — and the engine consults this wherever a literal name is matched, on every axis, so one override keeps /@alice and //@alice in agreement. :::name stays canonical (locators and reflection print it; an alias is a way in, never a way out), and name patterns (~(...), *) test the canonical name only.

fn has_trait(&self, node: NodeId, name: &str) -> bool

Whether node bears name as a trait, as written in a <...> clause. The default is membership in traits; an adapter may declare aliases the way answers_to does for names — a model file’s alias <chunk> <block>; or alias <s/^tag://i>; — and the engine consults this wherever a written trait is matched. Reflection and serialization print the canonical traits only.

fn ref_label(&self, _node: NodeId, _property: &str) -> Option<String>

The relation name a resolution edge from node via property carries — html’s rel attribute. None falls back to the property name as the edge label.

fn document_by_ref(&self, _id: &str) -> Option<NodeId>

The document this arbor holds under an external-reference identifier (a URL, fragment-stripped), when it can answer without the host’s registration: the landing rung of --> after the mounted-document table misses. An adapter over a store of pages answers by URL lookup; the engine performs no IO either way.

fn reverse_resolve( &self, _node: NodeId, _property: Option<&str>, _hint: Option<&str>, ) -> Option<Vec<NodeId>>

Every node whose property (or, when None, own reference property) resolves to node, in document order — the reverse index behind <--. None keeps the engine’s walk of the whole arbor; Some is the adapter’s complete answer.

fn descendants_named( &self, _node: NodeId, _name: &str, ) -> Option<Vec<(NodeId, usize)>>

The descendants of node that answer to name, each with its depth (children are depth 1), in document order — the fast path behind //name for an adapter whose substrate indexes names. None keeps the engine’s walk; Some must be exactly what the walk would find.

fn select(&self, from: NodeId, plan: &StepPlan<'_>) -> Option<Selection>

The search fast path: the nodes one step reaches from from under plan, answered from an index — None keeps the engine’s walk (the default folds children_named, descendants_named and descendants_where: select_by_hooks). Some must be exactly what the walk would find for the conjuncts it marks applied, under the engine’s standard equality; the engine tests the remaining conjuncts and every other predicate on the nodes returned. A child plan’s answer is never re-filtered by name (the adapter owns the name test, as children_named does); a descendant plan’s nodes still pass the nameless tests.

fn select_plan( &self, _scope: &[&str], _plan: &StepPlan<'_>, ) -> Option<Vec<bool>>

The dry-run form of select: which of the plan’s conjuncts the adapter would apply, decided from its catalog alone (a column’s existence and type, an index’s presence), with no node in hand. scope is the literal path from the root to the step’s base (["invoices"] for /invoices/*[...]), empty at the root. None when the adapter cannot say — the static plan then counts every conjunct as the engine’s.

fn count(&self, _from: NodeId, _plan: &StepPlan<'_>) -> Option<usize>

How many nodes one step reaches from from under plan, without materializing them — @| count right after the step asks before walking. None walks; Some must be exactly the walk’s count for the conjuncts the plan carries (every one of them, or none: a count cannot be partially applied). SQL: SELECT COUNT(*); a document index: the bucket’s length.

fn prefetch_properties(&self, _nodes: &[NodeId], _names: &[&str])

A batching hint, the projection twin of prefetch_links: the engine is about to read the properties names of every node in nodes (the next stage projects them). A remote adapter fetches the columns for the frontier in one round trip; semantics are unchanged either way.

fn prefetch_links(&self, _nodes: &[NodeId], _dir: LinkDir)

A batching hint: the engine is about to follow dir crosslinks from every node in nodes. An adapter backed by a store may fetch them in one statement; semantics are unchanged either way.

fn quantifier_bound(&self) -> usize

The quantifier bound N_max: the depth to which open-ended path quantifiers (+, *, {m,}) expand, and the ceiling of any explicit {m,n} (the effective upper bound is min(n, N_max)). An adapter whose natural structures run deep may raise it; a run may override it (Run::quantifier_bound, the CLI’s --quantifier-bound). The shell gate and the invocation instant are the run’s alone (Run::allow_shell, Run::now).

fn provenance(&self, _node: NodeId) -> Provenance

The data provenance of node — the :::source / :::instant / :::dpid / :::provenance core-metadata keys. Empty by default: an adapter answers only the components its substrate genuinely records (never the invocation clock); wrapper adapters fill missing components from what they know — the mount its target, a graft its outer leaf, a model its derivation — and forward the rest inward, so resolution is nearest-ancestor per component.

fn has_node(&self, node: NodeId) -> bool

Whether node is an id this adapter minted — the guard behind the node-id anchor ((!N)), which may name an id from an earlier run (ids are minted deterministically: the same source, the same ids). The default probes the id through parent and name and treats a panic as absence, so an adapter that indexes by id needs no override; one that can answer cheaply should.

fn provenance_list(&self, node: NodeId) -> ProvenanceList

The data provenance of node as a list — every source the substrate records for the datum, and how many it elides. The default is the singular answer as a one-entry list; an adapter whose substrate records several (kaiv’s ?a;b;+N) overrides it, and a wrapper layers each entry.

fn unit_scale(&self, expr: &str) -> Option<(f64, String)>

The scale of a unit expression — (factor, canonical SI-base expansion) — for the unital reading’s criterion text (spec: The Quantital Fragment). The default answers from the engine’s frozen built-in table; a unit-aware adapter (kaiv) overrides it to include the mounted document’s own custom units, so [::range < '50kellicam'] resolves through the document’s .!units imports.