Skip to main content

Module lifecycle

Module lifecycle 

Source
Expand description

S3 Lifecycle execution — per-bucket rule evaluation + manager skeleton (v0.6 #37).

AWS S3 Lifecycle attaches a list of rules to a bucket; each rule may request that S3

  1. Expire an object once its age (or the calendar date) crosses a threshold (Expiration { Days | Date }),
  2. Transition an object to a different storage class (Transition { Days, StorageClass }STANDARD_IA, GLACIER_IR, …),
  3. Expire noncurrent versions in a versioning-enabled bucket (NoncurrentVersionExpiration { NoncurrentDays }).

Until v0.6 #37 the matching PutBucketLifecycleConfiguration / GetBucketLifecycleConfiguration / DeleteBucketLifecycle handlers in crates/s4-server/src/service.rs were pure passthroughs (the s3s framework’s default backend stored them but nothing read the rules). This module owns the in-memory configuration store + the rule evaluator that decides, for any single object, whether an action should fire right now.

§responsibilities (v0.6 #37)

  • in-memory bucket -> LifecycleConfig map with JSON snapshot round-trip (mirroring versioning.rs / object_lock.rs / inventory.rs’s shape so --lifecycle-state-file is a one-line addition in main.rs).
  • per-bucket action counters (actions_total) — bumped by the future scanner when an Expiration / Transition / NoncurrentExpiration action is taken, surfaced via Prometheus (s4_lifecycle_actions_total, see metrics.rs).
  • LifecycleManager::evaluate — given one (bucket, key, age, size, tags) tuple, walk the bucket’s rules in declaration order and return the first matching action. Returns None when no rule matches (or when the matching rule is Disabled).
  • evaluate_batch — batched form for the test path: walks a slice of (key, age, size, tags) tuples and returns the (key, action) pairs that should fire. The actual backend invocation (S3.delete_object / metadata rewrite) is the caller’s job.

§scope limitations (v0.6 #37)

  • Background scanner is a skeleton only. main.rs’s --lifecycle-scan-interval-hours flag spawns a tokio task that logs the bucket list and stamps a “would-have-run” marker; walking the source bucket via list_objects_v2 and actually invoking delete_object / metadata rewrite for each evaluated action is deferred to v0.7+. Wiring the scheduler to walk a real bucket end-to-end requires a back-reference from the scheduler into S4Service for the list_objects_v2 walk and that reshuffle is out of scope for this issue. The crate::S4Service::run_lifecycle_once_for_test entry covers the in-memory equivalent so the unit + E2E tests exercise the evaluator end-to-end.
  • AbortIncompleteMultipartUpload is parsed and stored on the LifecycleRule (so PutBucketLifecycleConfiguration round-trips the field) but not enforced — multipart abort sweeping is a separate scanner that lives next to the multipart upload manager (v0.7+).
  • expiration_date (calendar date) is supported in the evaluator: a rule with expiration_date past now fires Expiration immediately. Same wire form as AWS S3.
  • Multi-instance replication. All state is single-instance in-memory; --lifecycle-state-file <PATH> provides restart recovery via JSON snapshot, matching the --versioning-state-file shape.
  • Object Lock interplay: the evaluator does NOT consult the ObjectLockManager directly (the evaluator API is object-tags-and-size only); the scanner caller is expected to skip locked objects — see the evaluate_batch_skips_locked test for the canonical pattern. Locking always wins over Lifecycle.
  • Versioning interplay: the evaluator treats noncurrent versions as a separate input — pass is_noncurrent = true to LifecycleManager::evaluate_with_flags for noncurrent version expiration matching. The legacy evaluate shorthand defaults is_noncurrent = false (current version) so existing call sites stay one-liners.

Structs§

EvaluateFlags
Flags for LifecycleManager::evaluate_with_flags. Default is “current-version object, evaluator picks Utc::now() for the date comparison”. Tests override now for determinism.
LifecycleConfig
Per-bucket lifecycle configuration (ordered list of rules — first match wins, matching AWS S3 semantics).
LifecycleFilter
Per-rule object filter. AWS S3 represents the filter as one of Prefix, Tag, ObjectSizeGreaterThan, ObjectSizeLessThan, or And (= AND of any subset of those predicates). For internal storage we flatten the “And” form into a struct of optional fields plus a vector of (key, value) tags — every present field must match (logical AND). An empty filter (all fields None / empty tags) matches every object in the bucket.
LifecycleManager
Per-bucket lifecycle configuration manager.
LifecycleRule
One lifecycle rule. AWS S3’s LifecycleRule flattened into the subset the v0.6 #37 evaluator handles. id is the operator-supplied label and makes Get / Put round-trips non-lossy.
MultipartUploadCandidate
v0.8.3 #69: one in-flight multipart upload the lifecycle scanner considers for abort. Mirrors the (subset of) MultipartUpload fields the rule evaluator needs (key, upload_id, initiated). tags is kept in the shape the existing object-path evaluator uses (Vec<(String, String)>) so a future enhancement that surfaces upload-time tags from MultipartStateStore can flow through the same filter check without API churn — AWS S3 itself does not attach tags to in-flight multipart uploads, so for the scanner-driven path the slice is always empty (the filter’s prefix / size predicates still apply via LifecycleFilter::matches, passing size = 0).
ScanReport
Per-invocation scanner counters returned by run_scan_once. Useful for tests, the --lifecycle-scan-interval-hours log line, and any future /admin/lifecycle/scan introspection endpoint. Operators see the same numbers via Prometheus (s4_lifecycle_actions_total{action="expire"|"transition"}).
TransitionRule
A single transition step (object age threshold + target storage class). days is days since the object was created. AWS S3 also accepts Date for transitions but Lifecycle deployments overwhelmingly use Days; the Date form is omitted here on purpose to keep the evaluator narrow (operators wanting calendar transitions can synthesise a one-shot rule at the cadence of their scanner).

Enums§

LifecycleAction
The action a single rule wants to take right now for a candidate object.
LifecycleStatus
Whether a rule is currently being applied. Mirrors AWS S3 ExpirationStatus ("Enabled" / "Disabled").

Functions§

evaluate_batch
Test-driven scan entry: walks a list of EvaluateBatchEntry tuples and produces (key, action) pairs for every object that should fire an action right now. The actual backend invocation (S3.delete_object / metadata rewrite) is the caller’s job. Used by both unit tests and the E2E test in tests/roundtrip.rs; the future background scanner will reuse the same entry once the bucket-walk is wired through the backend.
run_scan_once
Walk every bucket that has a lifecycle configuration attached, list its objects via list_objects_v2 (continuation-token pagination), and for each object evaluate the rule set + execute the matching Expiration / Transition action. Object-Lock-protected objects are skipped (the Lock always wins over Lifecycle). Versioning chains are intentionally out of scope for v0.7 #45 — see the module-level limitations note.

Type Aliases§

EvaluateBatchEntry
One object the evaluator considers in a batch: (key, object_age, object_size, object_tags). Defined as a type alias so evaluate_batch / crate::S4Service::run_lifecycle_once_for_test don’t trip clippy’s type-complexity lint, and so callers building the list have a single canonical shape to reach for.