Module staging 

Staged bulk load (#528): stage a page to object storage, then have the warehouse pull it with its native bulk-load command (COPY / COPY INTO / load-from-URI / s3() table function).

Row-by-row / multi-row INSERT is dramatically slower than the native bulk path on every cloud warehouse — for Redshift and Snowflake the staged COPY is the documented, production path. This module holds the shared, I/O-light pieces every capable sink reuses:

• StagingSpec — the user-facing staging: config block.
• StagingLocation — a parsed s3:// / gs:// / az:// location.
• StagedFile — a written object’s coordinates (uri / rows / bytes).
• record → bytes serialization for jsonl / csv (+ optional compression), and run-scoped object-key planning.
• Sink::supports_staged_load — a defaulted, object-safe capability flag.

The object-store upload itself lives behind the crate staging Cargo feature (it pulls object_store), so faucet-core stays lightweight for connector authors who don’t need it. Each warehouse sink turns the StagedFiles into its own load SQL — those generators are pure functions in the individual sink crates and unit-tested there.

Structs

StageUploader

Uploads staged objects to an ObjectStore and cleans them up. The store is injected so tests drive it against object_store::memory::InMemory.

StagedFile

Coordinates of one object written to the stage.

StagingLocation

A parsed staging location: a scheme, a bucket/container, and a key prefix.

StagingSpec

The user-facing staging: config block, flattened onto a capable sink’s config. Opt-in: absent means the sink uses its ordinary INSERT path.

Enums

StagingCleanup

When staged objects are deleted after a load.

StagingCompression

Outer compression for text staging formats.

StagingFormat

Physical file format written to the stage.

StagingScheme

The object-store scheme of a StagingLocation.

Functions

build_object_store

Build the object store for a location from environment/ambient credentials (the warehouse itself reads the files with its own grant; this store is only how faucet writes them). Kept minimal — richer credential wiring is a follow-up; the default credential chains cover the common case.

serialize_records

Serialize a page of records to the staging format’s raw bytes (before compression). Pure — the upload path applies compression and writes.