Crate r2kit

Expand description

§r2kit — Cloudflare R2 object transfers for Rust

Ergonomic, safety-first Cloudflare R2 transfers for Rust.

r2kit handles the R2-specific details around the official AWS S3 SDK: the account endpoint, auto signing region, secret-safe presigned requests, resumable multipart sessions, and managed file uploads with bounded concurrency and exact retries.

Status: 0.1.0 is the initial crates.io release. The API is still evolving, but core object and multipart workflows are verified against live Cloudflare R2.

§Why r2kit?

R2-native setup: build a correctly configured client from three required environment variables instead of wiring the S3 endpoint and signing behavior yourself. Temporary credentials can add an optional session token.
Safe secret boundaries: credentials, upload IDs, and presigned URLs are redacted from Debug; exposing bearer values requires explicitly named APIs.
Transfer workflows included: use simple object operations, managed local file uploads, or a server-controlled presigned multipart protocol.
Recovery by design: snapshot, resume, reconcile, cancel, and clean up multipart uploads without inventing a persistence format.
No lock-in: access the underlying aws_sdk_s3::Client whenever an operation is intentionally outside r2kit’s scope.

§Quick start

Add r2kit and a Tokio runtime:

cargo add r2kit
cargo add tokio --features macros,rt-multi-thread

Create a bucket-scoped R2 token and set its S3 credentials:

export R2_ACCOUNT_ID="your-32-character-account-id"
export R2_ACCESS_KEY_ID="your-access-key-id"
export R2_SECRET_ACCESS_KEY="your-secret-access-key"
# export R2_SESSION_TOKEN="..." # only for temporary credentials
# export R2_JURISDICTION="eu"   # default, eu, us, or fedramp

Upload, inspect, download, list, and delete an object:

use r2kit::R2Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let bucket = R2Client::from_env()?.bucket("media")?;

    let uploaded = bucket
        .put_bytes("hello.txt", b"hello R2".to_vec())
        .await?;
    let metadata = bucket.head("hello.txt").await?;
    assert_eq!(metadata.etag(), uploaded.etag());

    let downloaded = bucket.get("hello.txt").await?;
    let bytes = downloaded.into_body().collect().await?.into_bytes();
    assert_eq!(bytes.as_ref(), b"hello R2");

    let page = bucket.list().prefix("hello").limit(100).send().await?;
    assert_eq!(page.objects().len(), 1);

    bucket.delete("hello.txt").await?;
    Ok(())
}

§Client configuration

The default jurisdiction uses https://<ACCOUNT_ID>.r2.cloudflarestorage.com. Buckets with a data-residency jurisdiction require the matching eu, us, or fedramp endpoint. A jurisdiction is not a bucket location hint and does not change the signing region, which remains auto.

Use explicit configuration when an application needs transport bounds or SDK retry control:

use std::time::Duration;

use r2kit::{R2Client, R2Config, R2Jurisdiction};

fn client() -> Result<R2Client, r2kit::ConfigError> {
    let config = R2Config::builder()
        .account_id("0123456789abcdef0123456789abcdef")
        .access_key_id("access-key-id")
        .secret_access_key("secret-access-key")
        .jurisdiction(R2Jurisdiction::Eu)
        .connect_timeout(Duration::from_secs(5))
        .read_timeout(Duration::from_secs(30))
        .operation_attempt_timeout(Duration::from_secs(45))
        .operation_timeout(Duration::from_secs(120))
        .sdk_max_attempts(3)
        .build()?;

    Ok(R2Client::new(config))
}

Timeouts must be non-zero, and the per-attempt timeout cannot exceed the total operation timeout. sdk_max_attempts includes the initial request. It controls ordinary AWS SDK operations; managed multipart UploadPart requests disable SDK retries and use ManagedMultipartBuilder::max_attempts as their exact limit.

Leaving a transport option unset preserves the AWS SDK default. For custom credential providers, HTTP clients, proxies, endpoint resolvers, or other advanced SDK behavior, construct aws_sdk_s3::Client yourself and pass it to R2Client::from_sdk. That escape hatch cannot verify the R2 endpoint, auto region, credentials, timeouts, or retry policy, so the caller owns those invariants.

§Choose the right API

Use case	API
Upload bytes already in memory	`Bucket::put_bytes`
Upload a known-length async body	`Bucket::put_stream`
Download without buffering the whole object	`Bucket::get`
Upload a local file with concurrency and retries	`Bucket::managed_multipart`
Let a browser or mobile client upload directly	`Bucket::presigned_multipart`
Resume a persisted upload session	`Bucket::resume_managed_multipart` or `resume_presigned_multipart`
Verify bucket existence and list permission at startup	`R2Client::validate_bucket` or `Bucket::validate_access`
Use an S3 operation not wrapped by r2kit	`R2Client::as_sdk`

Bucket selection is offline by default. Applications that prefer a fail-fast startup check can explicitly perform one read-only request:

#[tokio::main]
async fn main() -> Result<(), r2kit::Error> {
    let client = r2kit::R2Client::from_env()?;
    let bucket = client.validate_bucket("media").await?;

    assert_eq!(bucket.name(), "media");
    Ok(())
}

§Typed object metadata

Use ecosystem media and header types instead of assembling security-sensitive HTTP values by hand. Existing upload methods keep their metadata-free behavior; the options variants opt into metadata explicitly.

use std::time::Duration;

use r2kit::{CacheControl, ObjectUploadOptions, R2Client, mime};

#[tokio::main]
async fn main() -> Result<(), r2kit::Error> {
    let bucket = R2Client::from_env()?.bucket("media")?;
    let options = ObjectUploadOptions::builder()
        .content_type(mime::IMAGE_JPEG)
        .cache_control(
            CacheControl::new()
                .with_public()
                .with_max_age(Duration::from_secs(3_600)),
        )
        .content_disposition("attachment; filename=cat.jpg")
        .content_language("en")
        .custom_metadata("tenant-id", "tenant-42")
        .build();

    bucket
        .put_bytes_with_options("photos/cat.jpg", vec![], options)
        .await?;
    Ok(())
}

For a presigned single PUT, typed metadata becomes part of the signature. The uploader must replay every header in PresignedRequest::required_headers exactly, and the bucket CORS policy must allow those headers. Multipart metadata is applied by the trusted server during CreateMultipartUpload; individual UploadPart requests do not repeat it.

Custom metadata keys omit the x-amz-meta- prefix and are canonicalized to lowercase. r2kit accepts portable ASCII metadata keys and values, rejects case-insensitive duplicates, and validates R2’s 8,192-byte metadata limit before network I/O. Content languages are validated structurally as one or more comma-separated BCP 47 language tags; registry-level language policy remains an application concern.

The core crate does not infer a MIME type from a filename. Browser applications should validate and parse File.type; local-file applications may deliberately use an extension-based helper such as mime_guess when guessing is acceptable.

§Paginated listings and batch deletion

send() intentionally fetches one bounded listing page. Use into_pages() to follow continuation tokens automatically while preserving page boundaries and common prefixes. Stream extension methods require futures-util in the application.

use futures_util::TryStreamExt;
use r2kit::R2Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let bucket = R2Client::from_env()?.bucket("media")?;
    let pages: Vec<_> = bucket
        .list()
        .prefix("temporary/")
        .into_pages()
        .try_collect()
        .await?;

    let keys = pages
        .iter()
        .flat_map(|page| page.objects())
        .map(|object| object.key().to_owned());
    let deleted = bucket.delete_objects(keys).await?;
    for failure in deleted.failures() {
        eprintln!("could not delete {}: {:?}", failure.key(), failure.code());
    }
    Ok(())
}

delete_objects validates every key before deleting anything, sends sequential batches of at most 1,000 keys, and reports service-level failures per key. A request-level BatchDeleteError retains results from batches that had already completed.

§Managed file uploads

Managed uploads split a local file into R2-compatible parts, upload them in parallel, retry transient failures, emit monotonic progress, and complete or abort the remote session.

use r2kit::R2Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let bucket = R2Client::from_env()?.bucket("media")?;
    let result = bucket
        .managed_multipart("videos/demo.mp4")?
        .part_size_mib(16)
        .concurrency(4)
        .max_attempts(4)
        .on_progress(|progress| {
            eprintln!(
                "{}/{} bytes",
                progress.transferred_bytes(),
                progress.total_bytes()
            );
        })
        .upload_file("videos/demo.mp4")
        .await?;

    eprintln!("completed {} parts", result.part_count());
    Ok(())
}

The uploader owns the UploadPart retry policy. Network failures, HTTP 408, 429, and 5xx responses are retried with exponential full jitter capped at 30 seconds. Numeric Retry-After seconds and AWS-compatible x-amz-retry-after milliseconds raise the delay up to that cap. AWS SDK retries are disabled for that operation, so max_attempts is the exact request-attempt limit.

Each in-flight part is buffered in memory. Before opening the file or contacting R2, the builder verifies that part_size * concurrency fits the 256 MiB default part-buffer budget. Configure an intentional larger bound with max_buffered_bytes or max_buffered_mib; this controls payload buffers rather than all process or SDK overhead.

Failures trigger a best-effort abort by default. Use abort_on_error(false) to retain ManagedUploadError::snapshot() for a later resume. The source file must not change while an upload is running. r2kit compares its size, modification time, and file identity where supported before completion, but applications should still treat immutability as a caller-owned invariant.

§Cancellation

Cancellation is cooperative: signal it from another task and continue awaiting the upload so r2kit can abort the remote multipart session.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let bucket = r2kit::R2Client::from_env()?.bucket("media")?;
    let cancellation = r2kit::ManagedUploadCancellation::new();
    let signal = cancellation.clone();

    let upload = bucket
        .managed_multipart("videos/demo.mp4")?
        .cancellation_token(cancellation)
        .upload_file("videos/demo.mp4");

    signal.cancel();
    let error = upload.await.unwrap_err();
    assert!(matches!(error.error(), r2kit::Error::Cancelled));
    Ok(())
}

Dropping the future cannot perform asynchronous cleanup. Signal cancellation and keep awaiting it instead.

§Lifecycle policies and cleanup

Cloudflare R2 automatically aborts incomplete multipart uploads seven days after initiation by default. Treat that bucket lifecycle rule as a final safety net rather than the primary cleanup path: keep awaiting cooperative cancellation so r2kit can abort promptly. Verify that the default rule remains enabled, or configure a shorter interval when abandoned uploads should be reclaimed sooner.

§Direct browser and mobile uploads

The trusted server creates a multipart session and signs each part. The untrusted uploader receives short-lived bearer URLs but never receives the R2 access key or secret.

file_size is intentionally required for this server-controlled flow. A web client sends its File.size when requesting a new upload; the server must treat that value as untrusted. r2kit validates it before contacting R2, uses it to calculate the number of parts and exact final-part length, rejects plans over R2’s object or 10,000-part limits, and verifies the same plan before completion. If the trusted application is uploading a local path instead, use managed_multipart(...).upload_file(path): that API reads the size itself.

trusted server                browser/mobile                    Cloudflare R2
      | create session               |                                |
      | sign part requests ---------->                                |
      |                              | PUT parts with signed headers ->|
      |<--------- exact ETags -------|<-------------------------------|
      | reconcile + complete ---------------------------------------->|

use std::time::Duration;

use r2kit::{CompletionManifest, PartMd5, PartNumber, R2Client};

async fn sign_and_complete(
    file_size_from_browser: u64,
    part_md5_base64: &str,
) -> Result<(), r2kit::Error> {
    let bucket = R2Client::from_env()?.bucket("media")?;
    let upload = bucket
        .presigned_multipart("videos/demo.mp4")?
        .file_size(file_size_from_browser)
        .part_size_mib(5)
        .create()
        .await?;

    let part = upload
        .presign_part_with_md5(
            PartNumber::try_from(1)?,
            PartMd5::try_from(part_md5_base64)?,
            Duration::from_secs(15 * 60),
        )
        .await?;

    // Deliberate exposure boundary for sending the bearer request to the uploader.
    let _request = part.into_protocol_request()?;

    // After collecting and validating every uploader receipt:
    let remote = upload.reconcile().await?;
    if remote.is_complete() {
        let manifest: CompletionManifest = remote.into_completion_manifest()?;
        upload.complete_verified(manifest).await?;
    }
    Ok(())
}

For browser uploads, configure bucket CORS to allow Content-MD5 and expose ETag. The uploader must replay every signed header exactly. Multipart ETags are opaque completion identifiers, not whole-object content hashes.

§Persistence and feature flags

The default build has no optional features enabled.

Feature	Purpose
`serde`	Serialize versioned multipart session records, signed request DTOs, and uploader receipts
`tracing`	Emit secret-safe diagnostic events through the application’s existing `tracing` subscriber
`live-tests`	Compile the credential-gated Cloudflare R2 integration tests; not intended for applications

Enable Serde when a session or protocol DTO crosses a storage or JSON boundary:

[dependencies]
r2kit = { version = "0.1.0", features = ["serde"] }

MultipartSessionSnapshot::into_persistence_record() deliberately exposes a secret-bearing persistence value. Store it as securely as an API credential.

§Errors and observability

Known numeric constraints are rejected locally before file or network I/O and include the supplied and accepted values. This includes multipart part size, part number, part count, object size, concurrency, attempt count, list limit, single-request upload size, and presign expiry.

use r2kit::{Error, ValidationError};

fn inspect(error: Error) {
    match error {
        Error::Validation(ValidationError::PartSizeOutOfRange {
            provided,
            min,
            max,
        }) => eprintln!("part size {provided} must be within {min}..={max} bytes"),
        Error::Remote(remote) => eprintln!(
            "{} failed: {} (status {:?})",
            remote.operation(),
            remote.kind(),
            remote.status()
        ),
        other => eprintln!("{other}"),
    }
}

Remote failures are reduced to a stable ServiceErrorKind, operation name, and optional HTTP status. Raw AWS SDK errors are intentionally not retained because they may contain signed request details. Object GET and HEAD preserve the convenient Error::NotFound result.

Tracing is opt-in and disabled by default:

[dependencies]
r2kit = { version = "0.1.0", features = ["tracing"] }

The library emits events to target r2kit but never installs a subscriber. Events contain operation/category/status and bounded transfer settings only; bucket names, object keys, local paths, account IDs, credentials, upload IDs, presigned URLs, and signed headers are excluded.

§Security model

Credentials, upload IDs, and presigned URLs are redacted from Debug and error messages.
Presigned URLs remain bearer credentials. Authorize before issuing them, use short expirations, and never log them.
Deterministic input failures are validated before network requests whenever possible.
R2 multipart plans enforce parts from 5 MiB through R2’s effective maximum of 5 MiB below 5 GiB, at most 10,000 parts, equal non-final part sizes, and the effective multipart object limit.
Managed uploads use bounded concurrency, a configurable part-buffer memory budget, capped full-jitter retries, and an exact retry limit.
Applications still own authorization, rate limiting, CORS policy, and the lifecycle policy for abandoned uploads.

Report vulnerabilities through GitHub’s private security advisory flow. See SECURITY.md for scope and reporting guidance.

§Examples and API documentation

The runnable examples require R2_BUCKET and R2_KEY. The managed upload example additionally accepts the local file path as its first argument.

§Compatibility and scope

Minimum supported Rust version: 1.94.1.
Runtime: Tokio.
Backend: Cloudflare R2 through aws-sdk-s3.
License: MIT or Apache-2.0, at your option.

For 0.1, bucket administration, ACLs, tagging, versioning, object lock, folder sync, a CLI, and a custom SigV4 implementation are deliberately out of scope.

§Development

The repository pins its Rust toolchain and keeps Git hooks in version control. After cloning, enable the hooks once:

./scripts/install-git-hooks.sh

Run the offline quality suite:

cargo fmt --all -- --check
cargo clippy --all-targets --no-default-features -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-targets --all-features
cargo test --doc --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --all-features --no-deps
cargo package

Property, fuzz, live contract, and 64 MiB stress-test commands are documented in CONTRIBUTING.md. Live tests only use the dedicated bucket and prefix supplied by the test operator, and clean up completed objects and active multipart sessions.

Contributions are welcome when they simplify an R2 transfer workflow, enforce an R2 invariant, or improve recovery and observability without hiding the underlying SDK. Please read CONTRIBUTING.md before opening a pull request.

§License

Licensed under either of:

Apache License, Version 2.0 (LICENSE-APACHE)
MIT License (LICENSE-MIT)

at your option.

Re-exports§

pub use mime;

Structs§

BatchDeleteError: A request-level batch delete error with results from earlier batches.
Bucket: Operations scoped to one R2 bucket.
CacheControl: Cache-Control header, defined in RFC7234 with extensions in RFC8246
CompletedObject: Result metadata for a completed multipart object.
CompletionManifest: A canonical, duplicate-free completion manifest sorted by part number.
DeleteObjectFailure: One per-key failure returned by an R2 multi-object delete request.
DeleteObjectsResult: Aggregate result of one or more DeleteObjects requests.
DownloadedObject: A streaming object download and its response metadata.
ListObjectsBuilder: Builder for one bounded page of an R2 object listing.
ManagedMultipartBuilder: Configures a bounded, retrying multipart upload from a local file.
ManagedUploadCancellation: A cloneable signal for explicitly cancelling a managed multipart upload.
ManagedUploadError: Failure from a managed upload, including recoverable session state.
ManagedUploadProgress: A point-in-time progress update for a managed multipart upload.
ManagedUploadResult: Successful outcome of a managed multipart file upload.
Mime: A parsed mime or media type.
MultipartPartReceipt: Untrusted transport data reported by a direct multipart uploader.
MultipartReconciliation: Server-observed state of a live multipart upload.
MultipartSessionRecord: Versioned persistence DTO for resuming a multipart session.
MultipartSessionSnapshot: Persistable state needed to resume a presigned multipart upload.
MultipartUploadPartRequest: Serializable protocol DTO sent from a trusted signer to an uploader.
ObjectMetadata: Metadata common to downloaded and inspected objects.
ObjectPage: A single page from an R2 object listing.
ObjectSummary: One object returned by a bucket listing.
ObjectUploadOptions: Typed system metadata applied when an object is created.
ObjectUploadOptionsBuilder: Builds typed metadata for a newly uploaded object.
PartMd5: A validated Base64-encoded 128-bit MD5 digest for one multipart part.
PartNumber: A validated multipart part number in the range 1..=10_000.
PresignedMultipart: An active R2 multipart upload that can presign parts, complete, or abort.
PresignedMultipartBuilder: Builder for a new presigned multipart upload session.
PresignedPutObject: A presigned single-request upload with an exact expected body length.
PresignedRequest: An HTTP request signed for temporary direct access to R2.
PresignedUploadPart: A presigned upload request for one known multipart part.
PutObjectResult: Result metadata for a successful single-request upload.
R2Client: A configured Cloudflare R2 client.
R2Config: Configuration needed to connect an S3-compatible client to Cloudflare R2.
R2ConfigBuilder: Builder for R2Config.
SecretUrl: A presigned URL treated as a bearer credential.
ServiceError: A sanitized remote failure that never exposes an SDK error or signed request.
UploadedPart: A successfully uploaded multipart part.

Enums§

ConfigError: A configuration value required to create an R2 client was absent or invalid.
Error: An error returned by an r2kit operation.
R2Jurisdiction: Data-residency jurisdiction used by an R2 bucket.
ServiceErrorKind: Sanitized category of an AWS SDK or Cloudflare R2 request failure.
ValidationError: A machine-readable failure of caller-provided numeric limits.

Crate r2kit

Crate r2kit Copy item path

§r2kit — Cloudflare R2 object transfers for Rust

§Why r2kit?

§Quick start

§Client configuration

§Choose the right API

§Typed object metadata

§Paginated listings and batch deletion

§Managed file uploads

§Cancellation

§Lifecycle policies and cleanup

§Direct browser and mobile uploads

§Persistence and feature flags

§Errors and observability

§Security model

§Examples and API documentation

§Compatibility and scope

§Development

§License

Re-exports§

Structs§

Enums§

Crate r2kit