Crate faucet_cli 

faucet-cli

A config-driven runner for faucet-stream pipelines. Define a source, optional transforms, a sink, and (optionally) a state store in a YAML or JSON file, then run it with the faucet binary — no Rust code required.

The library half of this crate exposes the same building blocks the binary uses (config parsing, env interpolation, the connector registry) so that integrations and tests can reuse them.

Re-exports

pub use error::CliError;

pub use error::CliResult;

Modules

auth_catalog

The top-level auth: provider catalog.

backfill

faucet backfill — replay a bounded historical window of a pipeline (#282).

catalogcatalog

Data Movement Catalog (#279) — the CLI-side write path.

chunking

Pure range chunking shared by faucet backfill and the partition: block (#479). Boundary parsing, timezone/DST-correct time windows, integer-range splitting, and offset/limit splitting. No I/O.

cli

Argument parser shared by main.rs and the integration tests.

commands

Subcommand implementations. Each module owns its run(...) entry point and stays free of clap so the integration tests can drive it directly.

compose

Config composition front pre-pass: extends (base inheritance), profiles (named overlays selected via --profile/FAUCET_PROFILE), and !include (YAML fragment substitution). Runs BEFORE ${...} interpolation and the secrets pass, reusing crate::merge::merge_value.

config

Parsed pipeline.yaml / pipeline.json schema (matrix-aware).

conformance

Connector conformance scoring (#330).

discovery_matrix

Pure planning logic for the discovery-driven request matrix (#501).

dlq_replay

faucet dlq — inspect, replay, and discard dead-letter-queue envelopes.

env_config

Build a PipelineConfig from a snapshot of FAUCET_* environment variables.

env_loader

Resolve a .env file path and auto-discover a pipeline config file.

error

CLI-level error type. Wraps every failure mode the binary surfaces so main() can render a single, user-readable line per failure.

executor

Run a list of ExpandedNodes under a bounded-concurrency executor.

expand

Expand a parsed PipelineConfig into a flat list of ExpandedNodes ready for the executor to run.

init_template

Schema-driven YAML template emitter for faucet init.

interpolate

Two-stage interpolation for pipeline configs.

lineage_gluelineage

Glue between the CLI config and faucet-lineage: builds the shared emitter and maps resolved transform specs onto column-lineage ops.

livemetricscli-progress or cli-tui

Shared live-view metrics plumbing (recorder install + Prometheus-text sampler), compiled when either live-view feature is on. Shared live-metrics plumbing for faucet run — the in-process Prometheus recorder installer plus a pure Prometheus-text parser/aggregator.

mcpmcp

MCP (Model Context Protocol) server surface for faucet (issue #420).

merge

Deep-merge of serde_json::Value for matrix-row overrides.

notifynotify

Notification / incident-routing layer (#280).

obs

Shared observability setup (Prometheus + tracing) used by run and schedule. install_observability is idempotent, so calling this once per process is safe even though main.rs already installed a basic subscriber.

params

Typed pipeline parameters — the params: block and the ${param.NAME} interpolation namespace (#444).

partition

Generic range partitioning (#479) — split one matrix row into N independent invocations, each scoped to a chunk of a range via ${partition.*} tokens.

pipeline_test

faucet test — fixture-based, fully-offline pipeline testing (#210).

progresscli-progress

Inline live progress for faucet run (#385, cli-progress feature).

reconcile

Completeness reconciliation (#502).

registry

Feature-gated dispatch from a string type to a concrete connector.

registry_index

Connector registry index (#208): the discovery/distribution layer behind faucet search, faucet install, and faucet list --available.

replication

faucet replicate — consistent snapshot → CDC streaming handoff.

scaffold

Connector SDK scaffolding (#209): pure templates for faucet new connector.

scheduleschedule

faucet schedule — built-in cron scheduler.

schema_compose

Compose the top-level JSON Schema for a whole faucet.yaml config (#213).

secrets

Secrets-manager interpolation for the config layer (#125).

select

Runtime matrix-row selection — the composable selection model spanning four issues that resolve through one eligibility → narrowing → parents → skip formula:

serveserve

faucet serve — HTTP control plane (#127). Runs pipeline configs submitted over HTTP, reusing executor::run_expanded. Feature-gated on serve; structured like cli/src/schedule/. See docs/superpowers/specs/2026-05-30-faucet-serve-design.md.

sla

Data-freshness & volume SLA monitoring (#202).

state

Build a StateStore from a (kind, config) pair, with feature-gated backends for Redis and PostgreSQL.

templatestemplates

Pipeline template registry (#444) — register a parameterized config once, then trigger runs by {id, params}.

topology

Topology mode (issues #71 / #72): build and run a faucet_core::Topology from a config’s pipeline.nodes / edges block.

transforms

Compile YAML/JSON transform declarations into TransformStage values.

tuicli-tui

Live terminal UI for faucet run --tui (#203, cli-tui feature).

Functions

run_command

Dispatch a parsed Cli to the matching command. Public so custom hosts and integration tests can drive the exact same code path as run_main with a programmatically-built Cli (and a registry installed via PluginRegistry::install).

run_from_yaml_str

Convenience entry point for integration tests and custom hosts: parse a YAML config string, expand the matrix, and run all rows.

run_main

Entry point for a custom faucet binary that bundles third-party connectors.