Crate faucet_source_file 

Local file source for faucet-stream: JSON Lines, JSON, CSV, Excel, XML, Parquet, Avro and ORC from a path, a directory, a glob, or one http(s):// URL — the local counterpart of the object-store sources.

Formats resolve per file from the extension (through a .gz / .zst suffix) unless one is set; incremental mode reads only files that are new since the previous run, by modification time or by name. A directory or glob skips a file sink’s unfinished output: its scratch files and the swap area of an overwrite run that has not committed.

Experimental (PRINCIPLES.md §3): this block’s shape may change in a minor release; any change is called out in the changelog.

Re-exports

pub use config::FileSourceConfig;

pub use config::IncrementalBy;

pub use config::IncrementalSpec;

Modules

config

File source configuration.

http

http(s):// paths: one remote file, fetched with bounded retries.

plan

Which files a run reads, in which order — listing, the stable_for_secs guard, and the incremental bookmark. Pure apart from list_local.

Structs

FileSource

A local-filesystem (or single http(s):// file) source.

ParquetReadOptions

Parquet read options: the same parquet: block as the object-store sources. Parquet reader options (the parquet: block of an object-store source).

Enums

FileSourceFormat

The format choice — auto by extension, per file, or one fixed format. Shared with the file sink through faucet-common-file. Which format a file is in: resolved from its extension (auto) or fixed.

Constants

DESCRIPTOR

The connector’s static descriptor (RFC 0013): its capabilities, answered from config alone.

Functions

read_records

Read every record at path with every default — format resolved per file from its extension, compression from its suffix. The one-call read-back a test or a script wants; build a FileSource for anything more.