toolcraft-s3-kit

S3-compatible object storage utilities for the toolcraft ecosystem.

Features

• 🚀 S3-compatible object storage client
• 📦 Support for MinIO and other S3-compatible services
• 🔐 Secure credential management
• ⚡ Async/await support with Tokio
• 🧩 Easy integration with the toolcraft ecosystem

Installation

Add this to your Cargo.toml:

[dependencies]
toolcraft-s3-kit = "*"

Check the crates.io page for the latest version.

Quick Start

use toolcraft_s3_kit::S3Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create S3 client
    let client = S3Client::new(
        "http://localhost:9000",
        "minioadmin",
        "minioadmin",
        None, // Optional region
    )?;
    
    // List buckets
    let buckets = client.list_buckets().await?;
    for bucket in buckets {
        println!("Bucket: {}", bucket.name);
    }
    
    Ok(())
}

Core Features

S3 Client

The S3Client provides a high-level interface for interacting with S3-compatible storage:

• Bucket Operations: Create, list, and delete buckets
• Object Operations: Upload, download, list, and delete objects
• Presigned URLs: Generate time-limited URLs for object access
• Streaming: Support for streaming large files
• Error Handling: Comprehensive error types for all operations

Example Usage

use toolcraft_s3_kit::S3Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = S3Client::new(
        "http://localhost:9000",
        "minioadmin",
        "minioadmin",
        None,
    )?;
    
    // Create a bucket
    client.create_bucket("my-bucket").await?;
    
    // Upload an object
    let data = b"Hello, world!";
    client.put_object("my-bucket", "hello.txt", data.to_vec()).await?;
    
    // Download an object
    let downloaded = client.get_object("my-bucket", "hello.txt").await?;
    println!("Downloaded: {}", String::from_utf8_lossy(&downloaded));
    
    // Generate presigned URL
    let url = client.presigned_get_object("my-bucket", "hello.txt", 3600).await?;
    println!("Presigned URL: {}", url);
    
    Ok(())
}

Download Examples

use toolcraft_s3_kit::S3Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = S3Client::new(
        "http://localhost:9000",
        "minioadmin",
        "minioadmin",
        None,
    )?;
    
    // Download file to local filesystem
    let bytes_downloaded = client
        .download_file("my-bucket", "large-file.pdf", "/tmp/downloaded.pdf")
        .await?;
    println!("Downloaded {} bytes to /tmp/downloaded.pdf", bytes_downloaded);
    
    // Download file to memory
    let file_bytes = client
        .download_to_bytes("my-bucket", "document.docx")
        .await?;
    println!("Downloaded {} bytes to memory", file_bytes.len());
    
    // Read text file content
    let text_content = client
        .read_text_file("my-bucket", "config.json")
        .await?;
    println!("File content:\n{}", text_content);
    
    Ok(())
}

Upload Local File Example

use std::sync::Arc;
use toolcraft_s3_kit::{BucketClient, S3Client};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Arc::new(S3Client::new(
        "http://localhost:9000",
        "minioadmin",
        "minioadmin",
        None,
    )?);

    let bucket = BucketClient::new(Arc::clone(&client), "my-bucket");
    let uploaded = bucket
        .upload_local_file(
            "reports/weekly.csv",
            "/tmp/weekly.csv",
            Some("text/csv"),
        )
        .await?;

    println!("Uploaded {} bytes", uploaded);
    Ok(())
}

Config Example (URL / Key / Secret / Bucket)

use toolcraft_s3_kit::S3BucketConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Required:
    // - TOOLCRAFT_S3_ENDPOINT (or S3_ENDPOINT)
    // - TOOLCRAFT_S3_ACCESS_KEY (or S3_ACCESS_KEY)
    // - TOOLCRAFT_S3_SECRET_KEY (or S3_SECRET_KEY)
    // - TOOLCRAFT_S3_BUCKET (or S3_BUCKET)
    // Optional:
    // - TOOLCRAFT_S3_REGION (or S3_REGION)
    let cfg = S3BucketConfig::from_env()?;
    let bucket = cfg.build_bucket_client()?;

    bucket
        .upload_local_file("plan.md", "./plan.md", Some("text/markdown"))
        .await?;
    bucket
        .upload_local_file("status.md", "./reports/status.md", Some("text/markdown"))
        .await?;

    Ok(())
}

Advanced Features

Custom Configuration

use toolcraft_s3_kit::S3Client;

let client = S3Client::new(
    "https://s3.amazonaws.com",
    "your-access-key",
    "your-secret-key",
    Some("us-east-1"), // Specify region
)?;

Error Handling

The crate provides detailed error types for different failure scenarios:

use toolcraft_s3_kit::{S3Client, error::S3Error};

match client.get_object("bucket", "key").await {
    Ok(data) => println!("Success!"),
    Err(S3Error::NotFound) => println!("Object not found"),
    Err(S3Error::AccessDenied) => println!("Access denied"),
    Err(e) => println!("Other error: {}", e),
}

Supported S3 Operations

• Bucket Operations

  • create_bucket()
  • list_buckets()
  • delete_bucket()
  • bucket_exists()

• Object Operations

  • put_object()
  • upload_local_file() - Upload local file path to S3
  • upload_bytes() - Upload in-memory bytes to S3
  • get_object()
  • delete_object()
  • list_objects()
  • object_exists()
  • read_text_file() - Read text files from S3
  • download_file() - Download files to local filesystem
  • download_to_bytes() - Download files to memory

• Presigned URLs

  • presigned_get_object()
  • presigned_put_object()

Integration with toolcraft

This crate is designed to work seamlessly with other toolcraft components:

// Use with toolcraft-config for configuration management
use toolcraft_config::Config;
use toolcraft_s3_kit::S3Client;

let config = Config::from_file("config.toml")?;
let s3_endpoint = config.get_string("s3.endpoint")?;
let s3_access_key = config.get_string("s3.access_key")?;
let s3_secret_key = config.get_string("s3.secret_key")?;

let client = S3Client::new(&s3_endpoint, &s3_access_key, &s3_secret_key, None)?;

License

This project is licensed under the MIT License - see the LICENSE file for details.

Links

• Repository
• Documentation
• Crates.io