rustfs-workers 0.0.3

Workers for RustFS, providing background processing capabilities for tasks such as data synchronization, indexing, and more.
Documentation
  • Coverage
  • 0%
    0 out of 8 items documented0 out of 6 items with examples
  • Size
  • Source code size: 18.15 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 1.49 MB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 24s Average build duration of successful builds.
  • all releases: 24s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • Homepage
  • rustfs/rustfs
    22017 938 77
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • houseme

RustFS

RustFS Workers - Background Job Processing

📖 Overview

RustFS Workers provides a distributed background job processing system for the RustFS distributed object storage system. It handles asynchronous tasks such as data replication, cleanup, healing, indexing, and other maintenance operations across the cluster.

Note: This is a core submodule of RustFS that provides essential background processing capabilities for the distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features

🔄 Job Processing

  • Distributed Execution: Jobs run across multiple cluster nodes
  • Priority Queues: Multiple priority levels for job scheduling
  • Retry Logic: Automatic retry with exponential backoff
  • Dead Letter Queue: Failed job isolation and analysis

🛠️ Built-in Workers

  • Replication Worker: Data replication across nodes
  • Cleanup Worker: Garbage collection and cleanup
  • Healing Worker: Data integrity repair
  • Indexing Worker: Metadata indexing and search
  • Metrics Worker: Performance metrics collection

🚀 Scalability Features

  • Horizontal Scaling: Add worker nodes dynamically
  • Load Balancing: Intelligent job distribution
  • Circuit Breaker: Prevent cascading failures
  • Rate Limiting: Control resource consumption

🔧 Management & Monitoring

  • Job Tracking: Real-time job status monitoring
  • Health Checks: Worker health and availability
  • Metrics Collection: Performance and throughput metrics
  • Administrative Interface: Job management and control

📦 Installation

Add this to your Cargo.toml:

[dependencies]
rustfs-workers = "0.0.3"

🔧 Usage

📋 Requirements

  • Rust: 1.70.0 or later
  • Platforms: Linux, macOS, Windows
  • Network: Cluster connectivity required
  • Storage: Persistent queue storage recommended

🌍 Related Projects

This module is part of the RustFS ecosystem:

📚 Documentation

For comprehensive documentation, visit:

🔗 Links

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

📄 License

Licensed under the Apache License, Version 2.0. See LICENSE for details.