Skip to main content

Crate pomelo_s3

Crate pomelo_s3 

Source
Expand description

Generic S3 ObjectSource for pomelo-data — reads objects from any S3-compatible store (Cloudflare R2, AWS S3, MinIO, GCS-S3) over HTTPS with SigV4-presigned GETs. Endpoint is configurable, so nothing here is Cloudflare-specific; the container points it at R2, OSS users at anything.

pomelo-data stays pure (trait + LocalSource); cloud access lives here.

Structs§

S3Conn
Resolved S3/R2 connection details from the environment.
S3Source
An ObjectSource backed by an S3-compatible bucket.

Enums§

OutStore
The destination for an fmp-sync run: a local path or an S3/R2 bucket. Both implement ObjectSink/ObjectSource, so sync_into writes a byte-identical data-layout.md tree either way (the S3 variant prepends an optional key prefix from the URL path).

Functions§

resolve_s3_conn
Resolve S3/R2 credentials + endpoint from the environment, trying the S3_ prefix first (Cloudflare R2 / static keys) then AWS_ (AWS S3, incl. IAM-role temporary credentials via AWS_SESSION_TOKEN). Per prefix P it reads {P}ACCESS_KEY_ID, {P}SECRET_ACCESS_KEY, optional {P}SESSION_TOKEN, {P}ENDPOINT/{P}ENDPOINT_URL (AWS: derived from {P}REGION if unset), and {P}REGION (default auto). get is injected so the chain is unit-testable. A different deployment prefix should be mapped onto S3_*/AWS_* in the env.