Crate s3rm_rs 

s3rm-rs

s3rm-rs is a fast Amazon S3 object deletion tool. It can be used to delete objects from S3 buckets with powerful filtering, safety features, and versioning support.

Features

• High Performance: Parallel deletion using S3 batch API (up to 1000 objects per request) with configurable worker pools (1–65,535 concurrent workers).
• Flexible Filtering: Regex patterns on keys/content-type/metadata/tags, size ranges, time ranges, and Lua script-based custom filtering.
• Safety First: Dry-run mode, confirmation prompts, force flag, max-delete threshold.
• Versioning Support: Delete markers, all-versions deletion for versioned buckets.
• Library-First: All CLI features available as a Rust library for programmatic use.
• s3sync Compatible: Reuses s3sync’s proven infrastructure (~90% code reuse).

Architecture

s3rm-rs uses a streaming pipeline architecture:

ObjectLister → [Filter Stages] → ObjectDeleter Workers (MPMC) → Terminator

All core functionality resides in this library crate. The CLI binary is a thin wrapper that parses arguments, builds a Config, and runs a DeletionPipeline.

Quick Start (Library Usage)

[dependencies]
s3rm-rs = "1"
tokio = { version = "1", features = ["full"] }

The easiest way is build_config_from_args — pass CLI-style arguments and get a ready-to-run Config:

use s3rm_rs::{build_config_from_args, DeletionPipeline, create_pipeline_cancellation_token};

#[tokio::main]
async fn main() {
    // Same arguments you would pass to the s3rm CLI.
    let config = build_config_from_args([
        "s3rm",
        "s3://my-bucket/logs/2024/",
        "--dry-run",
        "--force",
    ]).expect("invalid arguments");

    let token = create_pipeline_cancellation_token();
    let mut pipeline = DeletionPipeline::new(config, token).await;
    // The pipeline sends real-time stats to a channel for progress reporting.
    // Close the sender if you aren't reading from get_stats_receiver(),
    // otherwise the channel fills up and the pipeline stalls.
    pipeline.close_stats_sender();

    pipeline.run().await;

    // --- Error checking ---
    if pipeline.has_error() {
        if let Some(messages) = pipeline.get_error_messages() {
            for msg in &messages {
                eprintln!("Error: {msg}");
            }
        }
        std::process::exit(1);
    }

    let stats = pipeline.get_deletion_stats();

    println!("Deleted {} objects ({} bytes)",
        stats.stats_deleted_objects, stats.stats_deleted_bytes);
}

You can also build a Config with Config::for_target, but note that command-line validation checks are not performed (e.g. conflicting flags, rate-limit vs batch-size). Prefer build_config_from_args when possible:

let mut config = Config::for_target("my-bucket", "logs/2024/");
config.dry_run = true;          // preview without deleting
config.worker_size = 100;       // more concurrent workers
config.max_delete = Some(5000); // stop after 5 000 deletions

Lua Scripting

Lua filter and event callbacks can be registered via script paths in the Config:

config.filter_callback_lua_script = Some("path/to/filter.lua".to_string());
config.event_callback_lua_script = Some("path/to/event.lua".to_string());

Lua scripts run in a sandboxed VM by default (no OS or I/O library access). Use allow_lua_os_library and allow_lua_unsafe_vm on Config to relax restrictions.

For more information, see the s3sync documentation as s3rm-rs shares the same Lua integration.

Re-exports

pub use config::Config;

pub use config::args::CLIArgs;

pub use config::args::build_config_from_args;

pub use config::args::parse_from_args;

pub use types::DeletionStatistics;

pub use types::DeletionStats;

pub use types::DeletionStatsReport;

pub use types::S3Object;

pub use types::S3Target;

pub use types::error::S3rmError;

pub use types::error::exit_code_from_error;

pub use types::error::is_cancelled_error;

pub use types::DeletionError;

pub use types::DeletionEvent;

pub use types::DeletionOutcome;

pub use types::token::PipelineCancellationToken;

pub use types::token::create_pipeline_cancellation_token;

pub use types::event_callback::EventCallback;

pub use types::event_callback::EventData;

pub use types::event_callback::EventType;

pub use types::filter_callback::FilterCallback;

pub use callback::event_manager::EventManager;

pub use callback::filter_manager::FilterManager;

Modules

callback

Callback management for the deletion pipeline.

config

types

Structs

DeletionPipeline

The core deletion pipeline orchestrator.