Module config 

Parsed pipeline.yaml / pipeline.json schema (matrix-aware).

Top-level shape:

version: 1
name: optional-human-name
pipeline:               # required — full base config
  source: { type, config }
  sink:   { type, config }
  transforms: [...]
  state:  { type, config }
matrix:                 # optional — omitted or empty == one anonymous row
  - id: <string>
    parent: <id>
    parent_key: <jsonpath>   # default "id"
    source: { ... }     # partial override, deep-merged into pipeline.source
    sink:   { ... }
    transforms: [...]   # row-level transforms, appended after pipeline + source layers
    state:  { ... }     # if Some, replaces pipeline.state wholesale
execution:              # optional
  max_concurrent: <usize>
  on_error: continue|stop

The wire format is intentionally loose: every connector keeps its own config schema, and the CLI threads a serde_json::Value through to the connector’s serde::Deserialize impl. That keeps this struct stable as new fields are added to individual connectors without needing CLI work.

Structs

CircuitBreakerSpec

Circuit-breaker tuning.

CompletenessClaim

“For these column values, this fetch returns all the records” (#478).

ConnectorSpec

A { type, config } block, the universal shape for both sources and sinks.

DiscoverSpec

A discovery dimension’s source + projection (#501). See MatrixRow::discover.

DlqSpec

DLQ configuration block under pipeline.dlq:.

EdgeSpec

A topology edge (issue #71).

ExecutionSpec

Execution-time controls.

JoinSide

One side of a JoinSpec: the incoming edge label plus the dotted key path.

JoinSpec

The join: node configuration (issue #72).

MatrixRow

One row of the matrix: block.

ObservabilitySpec

Top-level observability block: Prometheus scrape endpoint and tracing level.

OtelSpec

OTLP export block under observability.otel:.

PartialConnector

A partial connector override carried by a matrix row. Both type and config are optional so rows can swap the kind, override only the inner config, or both. ref: (optional) picks which named template under pipeline.sources / pipeline.sinks this row instantiates; when absent, the row inherits the legacy singular pipeline.source / pipeline.sink (registered internally as a template named default).

PipelineConfig

Top-level pipeline definition.

PipelineSpec

The base pipeline definition. Each matrix row is resolved against the template catalogs below; the singular source / sink fields are the legacy way to declare a single template (internally named default).

PoisonSpec

Poison-pill (per-row) policy.

PrometheusSpec

Configuration for the Prometheus metrics HTTP endpoint.

ResilienceSpec

User-facing resilience: config. Maps to faucet_core::ResiliencePolicy.

RetrySpec

Retry/backoff tuning.

RunInputs

Resolve load-time ${env:} / ${file:} / ${secret:} directives in a composed config document after parsing rather than on the raw text.

SelectionSpec

Matrix-row selection policy (#377). Currently a single knob: include_parents. Extensible for future selection-model settings.

ShardingSpec

Source-shard distribution settings for clustered (Mode B) execution.

StateStoreSpec

State-store backend selector.

TracingSpec

Tracing / log-level configuration.

TransformSpec

A single transform declaration.

Enums

BackoffSpec

Backoff growth shape (config spelling of faucet_core::BackoffKind).

IncludeParents

Policy for pulling a selected row’s parent: / depends_on: ancestors into the run set when they are not independently selected (#377). The only mechanism that decides ancestor inclusion. Default Off (strict).

NodeSpec

A single topology node (issue #71). The kind discriminator selects the node type; the remaining fields are kind-specific.

OnBatchErrorSpec

Mirrors faucet_core::OnBatchError but with JsonSchema derived and Deserialize accepting the YAML/JSON shape. Converted to the core type during executor::build_dlq_config.

OnError

Failure-handling policy across the matrix.

OnMissing

Action for destination rows inside a completeness scope that a run did not write (#478).

PoisonActionSpec

Terminal action for a poison row (config spelling of faucet_core::PoisonAction).

SourceStatus

Readiness ladder for a matrix row’s source (#371). A single ordered axis answering “when does this row run?”. Default (when absent) is Active.

Functions

parse_with_extension

Convenience: parse a config from text using a synthetic path so the right parser is selected. Used by tests and the validate --stdin flow.