Crate rust_task_queue

Rust Task Queue Framework

A high-performance, Redis-backed task queue system with auto-scaling capabilities designed for use with async Rust applications.

Features

• Redis-backed broker for reliable message delivery
• Auto-scaling workers based on queue load
• Task scheduling with delay support
• Multiple queue priorities
• Retry logic with configurable attempts
• Task timeouts and failure handling
• Metrics and monitoring
• Actix Web integration (optional)
• Automatic task registration (optional)

Quick Start

Basic Usage

use rust_task_queue::prelude::*;
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize, AutoRegisterTask)]
struct MyTask {
    data: String,
}

#[async_trait::async_trait]
impl Task for MyTask {
    async fn execute(&self) -> Result<Vec<u8>, Box<dyn std::error::Error + Send + Sync>> {
        println!("Processing: {}", self.data);
        use serde::Serialize;
        #[derive(Serialize)]
        struct Response { status: String }
        let response = Response { status: "completed".to_string() };
        Ok(rmp_serde::to_vec(&response)?)
    }
     
    fn name(&self) -> &str {
        "my_task"
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let task_queue = TaskQueue::new("redis://localhost:6379").await?;
     
    // Start workers
    task_queue.start_workers(2).await?;
     
    // Enqueue a task
    let task = MyTask { data: "Hello, World!".to_string() };
    let task_id = task_queue.enqueue(task, "default").await?;
     
    println!("Enqueued task: {}", task_id);
    Ok(())
}

Automatic Task Registration

use rust_task_queue::prelude::*;
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize, Default, AutoRegisterTask)]
struct MyTask {
    data: String,
}

#[async_trait::async_trait]
impl Task for MyTask {
    async fn execute(&self) -> Result<Vec<u8>, Box<dyn std::error::Error + Send + Sync>> {
        println!("Processing: {}", self.data);
        use serde::Serialize;
        #[derive(Serialize)]
        struct Response { status: String }
        let response = Response { status: "completed".to_string() };
        Ok(rmp_serde::to_vec(&response)?)
    }
     
    fn name(&self) -> &str {
        "my_task"
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Task is automatically registered!
    let task_queue = TaskQueueBuilder::new("redis://localhost:6379")
        .auto_register_tasks()
        .initial_workers(2)
        .build()
        .await?;
     
    let task = MyTask { data: "Hello, World!".to_string() };
    let task_id = task_queue.enqueue(task, "default").await?;
     
    println!("Enqueued task: {}", task_id);
    Ok(())
}

Re-exports

pub use inventory;auto-register

pub use autoscaler::*;

pub use broker::*;

pub use cli::*;

pub use config::*;

pub use error::*;

pub use metrics::*;

pub use queue::*;

pub use scheduler::*;

pub use task::*;

pub use worker::*;

Modules

actixactix-integration

Actix Web integration helpers with comprehensive metrics endpoints

autoscaler

axumaxum-integration

Axum integration helpers with comprehensive metrics endpoints

broker

clicli

Consumer Helper Functions

config

Configuration management for the task queue framework

error

metrics

prelude

Convenience re-exports for common types and traits

queue

scheduler

task

tracing_utils

Comprehensive tracing utilities for the Rust Task Queue system

worker

Macros

create_axum_task_endpointaxum-integration

Helper macro for creating typed task endpoints for Axum

create_task_endpointactix-integration

Helper macro for creating typed task endpoints

create_worker_maincli

Macro to create a complete task worker binary with minimal code

create_worker_with_buildercli

Macro to create a task worker with custom configuration using the builder pattern

manual_register_task

register_task_with_name

register_tasks

timed_operation

Helper macro for timing operations with automatic tracing

traced_operation

Helper macro for creating traced spans around async operations

Structs

ComponentHealth

HealthStatus

Health check status for monitoring

TaskQueue

Main task queue framework

TaskQueueBuilder

Builder for configuring and creating TaskQueue instances

TaskQueueMetrics

Comprehensive metrics for monitoring

Attribute Macros

proc_register_task

Attribute macro for registering tasks with a custom name.

Derive Macros

AutoRegisterTask

Automatically register a task type with the task registry.