# faucet-source-singer
> **Support tier: Tier-2 / experimental.** Best-effort — correctness bugs are
> fixed, but breadth of testing and upstream-drift tracking are not guaranteed.
A [Singer](https://www.singer.io/) tap bridge source for
[faucet-stream](https://github.com/PawanSikawat/faucet-stream). It runs an
existing Singer **tap** executable and adapts its stdout message stream into
faucet records — so any of the hundreds of community taps can feed a faucet
pipeline.
## Honest trade-offs
- **It reintroduces a runtime dependency.** Most Singer taps are Python; a
pipeline using this source needs that interpreter (and the tap) installed on
the box. This is the one place faucet steps outside its "single static binary"
story — use a native faucet source when one exists.
- **Throughput is Singer-class, not faucet-class.** Records cross a process
boundary as newline-delimited JSON; expect tap-bound throughput, not faucet's
native streaming numbers.
- **Resume granularity depends on the tap.** faucet checkpoints at the tap's own
`STATE` messages; how coarse or fine that is (and whether re-emitted rows
overlap) is a property of the individual tap. Pair with an idempotent sink for
clean **effectively-once** (idempotent at-least-once) behavior.
## v0 scope
- **Single-stream.** Exactly the configured `stream` is emitted; RECORD messages
for other streams are ignored. Multi-stream fan-out is future work.
- Handles `RECORD`, `SCHEMA` (pass-through — faucet sinks infer schema from
records), and `STATE` (resume bookmark). `ACTIVATE_VERSION` / `BATCH` are
logged and skipped.
## Configuration
| `executable` | string | — (required) | Tap binary on `PATH` or an absolute path |
| `stream` | string | — (required) | The single stream to emit |
| `args` | string[] | `[]` | Extra args appended after faucet's `--config`/`--catalog`/`--state` |
| `tap_config` | object | `{}` | The tap's config (secret-resolved by faucet; written to a private temp file) |
| `catalog` | object | — | Singer catalog, passed as `--catalog` |
| `state_key` | string | `singer:{executable}:{stream}` | State-store key for the resume bookmark |
| `flush_on_state` | bool | `true` | Flush a page (and checkpoint) on every STATE message |
| `idle_timeout_secs` | int | — | Abort if no output arrives within this many seconds |
| `on_malformed` | `skip` \| `fail` | `skip` | What to do with a non-Singer output line |
## Catalog-driven stream selection
Most database and Meltano-SDK taps **sync nothing** unless the target stream is
marked `selected` in the catalog — a catalog passed through verbatim is a silent
no-op. Parent-keyed taps go further: `tap-github`'s `issues` stream only syncs
when its parent `repositories` stream is *also* selected, even though faucet
emits just `issues`.
`faucet init --source singer --discover --executable <tap> --stream <name>` runs
the tap's discovery and writes a catalog with `<name>` — **and any inferable
parent streams** — marked `selected` (both the stream-level flag and the
`breadcrumb: []` metadata). When the tap doesn't express a parent relationship in
its catalog, `faucet init` prints a warning listing the other streams so you can
select a parent manually. Extraction may be multi-stream (parents are pulled to
satisfy the child), but faucet still **emits only the configured `stream`**.
> **`ACTIVATE_VERSION` / `FULL_TABLE` + an append sink can accumulate
> duplicates.** A tap doing full-table reloads re-emits every row each run; an
> append-only sink (jsonl/csv/stdout) keeps them all. Pair such taps with a
> **keyed sink** (`write_mode: upsert`, `key: [...]`) so re-emitted rows converge.
## Example
```yaml
version: 1
pipeline:
source:
type: singer
config:
executable: tap-github
stream: issues
tap_config:
access_token: ${env:GITHUB_TOKEN}
repository: PawanSikawat/faucet-stream
sink:
type: jsonl
config:
path: ./out/issues.jsonl
state:
type: file
config:
path: ./state
```
## License
Licensed under either of Apache-2.0 or MIT at your option.