Module stage 

Pipeline-level transform stages. A TransformStage wraps one of six shapes:

• TransformStage::Map holds an unchanged 1→1 RecordTransform.
• TransformStage::Filter is a predicate-based 1→0|1 stage.
• TransformStage::Explode expands an array field into 1→0..N output records.
• TransformStage::CdcUnwrap normalizes a CDC change-event envelope ({op, before, after, …}) into a flat row plus a marker field; DDL and truncate events are dropped (1→0|1).
• TransformStage::Custom is an Fn(Value) -> Vec<Value> closure escape hatch for library callers.
• TransformStage::PageFn is a Fn(Vec<Value>) -> Result<Vec<Value>, FaucetError> whole-batch closure for transforms that need the full page at once (SQL, sort, dedup, top-N). Not addressable from YAML.

apply_stages_to_page is the page-granular runner: per-record stages (Map/Filter/Explode/Custom) flat-map over each record in order, while PageFn stages receive and return the whole page slice. Declared order is preserved, so filter → sql → select works as expected. The observability wrapper crate::observability::instrumented_apply_stages wraps apply_stages_to_page and aggregates per-page metrics.

Structs

CdcUnwrapSpectransform-cdc-unwrap

Spec for TransformStage::CdcUnwrap. Normalizes a CDC change-event envelope ({op, before, after, …}) into a flat row plus a marker field, so a downstream upsert sink never needs to understand CDC. A 1→0|1 stage: DDL/truncate events (and non-delete events with no after image) are dropped.

CompiledCdcUnwraptransform-cdc-unwrap

Pre-compiled cdc_unwrap. pub so it can sit inside the pub CompiledStage::CdcUnwrap variant without triggering the private_interfaces lint — same shape as the already-pub CompiledExplode sibling that backs CompiledStage::Explode.

CompiledExplodetransform-explode

Pre-compiled explode: path parsed once, prefix resolved once.

CompiledFiltertransform-filter

Pre-compiled filter: path parsed once, value cloned once.

CompiledPath

Pre-parsed JSONPath. Bad syntax surfaces at compile_stage time, not per record. The implementation is a hand-rolled parser for the v1 subset (bare key, dot path, bracketed string key) — sufficient because we need to walk parent-container chains for explode mutation, which the jsonpath_rust::query() API doesn’t expose.

ExplodeSpectransform-explode

Spec for TransformStage::Explode. Fans one record out into N records based on the array at path. Compiled into a CompiledExplode at pipeline-build time; per-record work is one path resolve + N clones + N merges.

FilterSpectransform-filter

Predicate spec for TransformStage::Filter. Compiled into a CompiledFilter at pipeline-build time; per-record work is a single path resolve + comparison.

Enums

CompiledStage

Pre-compiled stage. Per-record work is just lookup + comparison + flat-map.

FilterOptransform-filter

Comparison operator for FilterSpec.

OnMissingtransform-explode

Behaviour when an ExplodeSpec’s path doesn’t yield a non-empty array. The default is Passthrough because silently dropping records is the worst failure mode for ETL pipelines — surfacing the record unchanged lets downstream stages decide.

PathSegment

TransformStage

One stage in a transform pipeline.

Functions

apply_stages

Per-record stage runner. Returns 0..N output records. Pure; no metrics.

apply_stages_to_page

Page-granular stage runner. Per-record stages (Map/Filter/Explode/ Custom) flat-map over the current page; PageFn stages transform the whole page at once. Preserves declared order, so filter → sql → select works.

compile_stage

Compile a TransformStage into its CompiledStage form.

Type Aliases

PageFnBox

Type alias for the page-level transform closure stored in TransformStage::PageFn and CompiledStage::PageFn.