cellos-export-s3 0.5.1

S3 ExportSink for CellOS — uploads per-cell evidence bundles to an S3 bucket for centralised audit retention.
Documentation

cellos-export-s3

ExportSink that uploads evidence artifacts via an S3 presigned PUT URL and writes logical s3://bucket/key receipts.

What it is

Implements cellos_core::ports::ExportSink. For each pushed artifact, the sink:

  1. Resolves a presigned PUT URL from the configured template (substituting {cell_id} and {artifact_name} if present, else using the URL exact).
  2. PUTs the artifact bytes via reqwest with bounded request and connect timeouts.
  3. Retries on transient errors with a fixed backoff (max_attempts, retry_backoff_ms).
  4. Returns an ExportReceipt with targetKind = S3 and a logical s3://<bucket>/<key> destination — even though the transport is HTTP, receipts stay logical so audit tooling speaks S3.

Selected in cellos-supervisor::composition::build_s3_transport_sink for named export targets in the cell spec whose targetKind is S3. The S3 sink is not the default — operators name an S3 target and the supervisor picks this sink for it.

What it does NOT do:

  • It does not sign requests with SigV4. Auth lives in the presigned URL; the supervisor never holds AWS credentials.
  • It does not list, delete, or presign — write-only PUT.
  • It does not perform multipart upload. One artifact, one PUT.
  • It does not look at AWS env vars (AWS_ACCESS_KEY_ID etc.) — the presigned URL is the entire authentication story.

Public API surface

Symbol Purpose
PresignedS3ExportSink The sink.
PresignedS3ExportSink::new(presigned_url, cell_id, bucket, key_prefix, region, ca_bundle, max_attempts, retry_backoff_ms) Constructor; rejects unparseable URLs.
DEFAULT_REQUEST_TIMEOUT_MS / ENV_REQUEST_TIMEOUT_MS 30 s default request timeout; overridable.
DEFAULT_CONNECT_TIMEOUT_MS / ENV_CONNECT_TIMEOUT_MS 10 s default connect timeout; overridable.
resolve_timeout_ms(env_var, default_ms) Pure helper.

Source: src/lib.rs.

Configuration

Per named S3 target <NAME> in the cell spec (__<NAME> is the upper-snake form of the target name):

Env var Description
CELLOS_EXPORT_S3_PRESIGNED_URL__<NAME> Presigned PUT URL (may contain {cell_id} / {artifact_name}).
CELLOS_EXPORT_S3_REGION__<NAME> Optional region hint; overrides the value declared in the spec.
CELLOS_EXPORT_S3_MAX_ATTEMPTS__<NAME> Total PUT attempts including the first try (≥ 1).
CELLOS_EXPORT_S3_RETRY_BACKOFF_MS__<NAME> Fixed delay between attempts (ms).

Global, applies to all targets:

Env var Default
CELLOS_EXPORT_S3_TIMEOUT_MS 30 000
CELLOS_EXPORT_S3_CONNECT_TIMEOUT_MS 10 000
CELLOS_CA_BUNDLE PEM CA bundle for private TLS PKI.

HTTP_PROXY / HTTPS_PROXY / NO_PROXY are honoured by reqwest. The client is never built without explicit timeouts.

A legacy fallback exists: if CELLOS_EXPORT_S3_PRESIGNED_URL__<NAME> is unset, the supervisor will read CELLOS_EXPORT_HTTP_BASE_URL__<NAME> instead and record a StartupConfigWarning (so CELLOS_STRICT_CONFIG=1 refuses to start until the operator switches to the canonical name).

Examples

Cell spec declares a named S3 target artifact-bucket:

export:
  targets:
    - name: artifact-bucket
      kind: s3
      bucket: my-cellos-evidence
      region: us-west-2

Operator wires it up:

export CELLOS_EXPORT_S3_PRESIGNED_URL__ARTIFACT_BUCKET="https://my-cellos-evidence.s3.us-west-2.amazonaws.com/{cell_id}/{artifact_name}?X-Amz-Signature=..."
export CELLOS_EXPORT_S3_MAX_ATTEMPTS__ARTIFACT_BUCKET=3
export CELLOS_EXPORT_S3_RETRY_BACKOFF_MS__ARTIFACT_BUCKET=500
cellos-supervisor --spec cell.yaml

Testing

cargo test -p cellos-export-s3

Tests exercise URL templating, retry logic, and the timeout-resolution helper without hitting a live S3 endpoint.

Related crates

  • cellos-export-local — local-disk default.
  • cellos-export-http — generic HTTP PUT for non-S3 endpoints.
  • cellos-supervisor — selects this sink in build_s3_transport_sink.
  • cellos-core — defines ExportSink, ExportReceipt, ExportReceiptTargetKind.

Operator note: tracing

reqwest can emit the presigned URL (whose query string carries the SigV4 X-Amz-Signature) and any session-token header at TRACE level. Every CellOS binary that initializes tracing wires cellos_core::observability::redacted_filter into the fmt layer so RUST_LOG=reqwest=trace cannot leak presigned credentials to stderr. Do not bypass this filter in custom tracing-init paths.

ADRs

  • ADR-0006 — evidence bundle is the 1.0 deliverable; S3 is the default multi-host destination.