Crate tabnas_transduce

Expand description

Streaming transducers over any tabnas parser.

A transducer decides what a source contributes to an output model; a renderer (the tabnas-render crate) decides how that model becomes text. This crate is the transducer side:

the source protocol JsonEvent (JsonEvents/1) and the push boundary Sink;
sources that produce it from a parsed value, from a live tabnas parse through the engine’s rule events, or from line-delimited input;
Selectors and the matcher and router that recognize selected scopes in one pass and materialize them under a byte limit;
the table protocol TableEvent (TableRows/1) and the standard metadata-first table transducer;
Limits, Metrics, AbortFlag and the stable failure Codes every stage reports.

Everything is synchronous and push-based: the parser calls the first sink from inside its own callback, and a slow writer at the end of the chain slows the parse at the start. That is the backpressure.

The protocol types (events, sinks, tables, failures, limits, selectors, datums, and the data on either side of a router and of scan-emit) are tabnas-alchemy’s shared types, tabnas_alchemy::shared, re-exported here at the paths they have always had. routers() answers this crate’s implementation of alchemy’s Routers, the interface a compiled alchemy program makes its routing stages through.

Re-exports§

pub use matcher::Hit;
pub use matcher::HitKind;
pub use matcher::Matcher;
pub use route::Router;
pub use routers::routers;
pub use routers::TransduceRouters;
pub use scan::ScanEmit;
pub use source::capability;
pub use source::walk_value;
pub use source::Guarded;
pub use source::ParserSource;
pub use source::Prune;
pub use source::Source;
pub use source::SourceMode;
pub use source::ValueSource;
pub use table_from_json::TableFromJson;

Modules§

datum: The protocol modules, which are alchemy’s shared types. The retained value type.
error: The protocol modules, which are alchemy’s shared types. Failures, with stable codes.
event: The protocol modules, which are alchemy’s shared types. JsonEvents/1: the source protocol every transducer consumes.
limits: The protocol modules, which are alchemy’s shared types. Limits, metrics and cancellation.
matcher: Shared-prefix matching of many selectors in one pass.
route: Captures: recognize selected scopes and deliver them, complete, in source order.
routers: This crate’s implementation of alchemy’s Routers: the interface a compiled alchemy program makes its routing stages through.
scan: scan-emit: declarative state evolution over a stream.
selector: The protocol modules, which are alchemy’s shared types. Selectors: reusable descriptions of where in a document to look.
sink: The protocol modules, which are alchemy’s shared types. The push boundary between stages.
source: Sources: where JsonEvents/1 come from.
table: The protocol modules, which are alchemy’s shared types. TableRows/1: one schema, ordered finite rows, completion.
table_from_json: The metadata-first table transducer: JsonEvents/1 in, TableRows/1 out.

Structs§

AbortFlag: A cancellation flag shared by the caller, the source and the stages.
BoundColumn: A column as the transducer binds it: the public label, and the source path projected from each row. Never crosses into a renderer.
Budget: The types on either side of a router, which are alchemy’s shared types. The byte budget one materialized capture may not exceed, named after the Limits field the failure reports.
CaptureSpec: The types on either side of a router, which are alchemy’s shared types. One capture: a tag for the consumer, the selector to match, the mode.
CountSink: A sink that counts events and drops them: the cheapest consumer, for measuring a source on its own.
DatumBuilder: Builds one Datum from the events of one value, under a byte limit.
Fail: A failure: a code, and what is known about where and why.
FnRoute: The types on either side of a router, which are alchemy’s shared types. A route sink made of a closure; end is a no-op.
FnSink: A sink made of a closure.
Limit: The limit a Code::ResourceLimitExceeded names.
Limits: Per-run limits. Every field is named in a failure as written here.
Metrics: Counters and high-water marks a run reports. Shared between stages through an Arc; every update is one relaxed atomic.
Number: A number as the source spelled it and as a machine value.
Path: A concrete location in a document.
PublicColumn: What a renderer knows about a column.
Selected: The types on either side of a router, which are alchemy’s shared types. One completed match.
Selector: A description of locations: the root, narrowed step by step.
Table: An owned recording of a table, for tests and small results.
TableBinding: A table transducer’s source binding.
Transition: The result of one step, which is one of alchemy’s shared types. The result of one step: the next state and what to emit for it.
TreeContract: A tree’s events, held to their contract in front of a sink that takes them as one (a render that writes a document from them): one root value, and in each object a key and then its value, each key once. A value walked from a parsed tree keeps it by construction. A parse streamed as it proceeds may not: it hands on a member its grammar reads twice (JSON’s {"a":1,"a":2}, whose value keeps the last) where a tree has one, and the rule-event adapter refuses most shapes it cannot follow but not every one a grammar can produce. A repeated key in one object is refused with DUPLICATE_MEMBER, and events no tree has (a value where a key is due, a key outside an object, a close with nothing open, a second root) with STREAMABILITY_UNKNOWN, each at the path of the object concerned; the event is not passed on, and what to do then is the host’s (aless falls back once to the parsed value when nothing has been written). Each open object keeps the keys it has had, dropped when it closes, so the cost is a lookup per key and the keys of the objects open at once, which a parse holds in its tree already.

Enums§

CaptureMode: The types on either side of a router, which are alchemy’s shared types. What a capture keeps of its match.
Cell: One projected value.
Code: The stable failure codes.
Datum: A retained JSON-like value.
Duplicates: How a builder treats a repeated member name.
Flow: What a stage wants next.
JsonEvent: One event of JsonEvents/1.
MissingPolicy: What to do when a row has no value at a column’s path.
OwnedJsonEvent: JsonEvent with its text owned: what a recorder keeps.
Schema: Where a table’s columns come from.
Segment: One step of a concrete path.
Step: One step of a selector.
TableEvent: One event of TableRows/1.

Constants§

VERSION: This crate’s version, as Cargo.toml declares it.

Traits§

RouteSink: The types on either side of a router, which are alchemy’s shared types. The consumer of a router’s matches.
Sink: A consumer of JsonEvents/1.
TableSink: A consumer of TableRows/1.

Functions§

column_from_meta: The standard mapping from a metadata descriptor to a column: the spec’s column-from-meta, reading title and a path of segments.
replay: Replay a recording into a sink, stopping where the sink stops.
walk_datum: Emit a datum as JsonEvents/1 (without the final End, so a datum can stand in for any part of a document).
write_json: Append a datum as compact JSON.
write_json_number: Append a number: its lexeme when known, else the shortest text that reads back as the same f64. A non-finite value has no JSON form and is written as null; renderers reject it before it gets here.
write_json_string: Append s as a JSON string literal, escaped as RFC 8259 requires: " and \\ escaped, control characters as \\uXXXX (with the short forms for \\b \\f \\n \\r \\t), everything else as itself.

Type Aliases§

CaptureId: Which selector matched: its position in the slice given to Matcher::new. One of alchemy’s shared types. Which selector matched: its position in the slice given to transduce’s Matcher::new.
ColumnMapper: Maps one metadata descriptor to a bound column.