celery-rs 0.6.1

📢 Project Status: This is a community-maintained fork of the original rusty-celery project. The original project became inactive, so we've taken over maintenance to ensure continued development and support for the Rust Celery ecosystem.

We welcome contributions from everyone regardless of your experience level with Rust. For complete beginners, see HACKING_QUICKSTART.md.

If you already know the basics of Rust but are new to Celery, check out the Rusty Celery Book or the original Python Celery Project.

Quick start

Define tasks by decorating functions with the task attribute.

use celery::prelude::*;

#[celery::task]
fn add(x: i32, y: i32) -> TaskResult<i32> {
    Ok(x + y)
}

Create an app with the app macro and register your tasks with it:

let my_app = celery::app!(
    broker = AMQPBroker { std::env::var("AMQP_ADDR").unwrap() },
    tasks = [add],
    task_routes = [
        "*" => "celery",
    ],
).await?;

Then send tasks to a queue with

my_app.send_task(add::new(1, 2)).await?;

And consume tasks as a worker from a queue with

my_app.consume().await?;

Examples

The examples/ directory contains:

• a simple Celery app implemented in Rust using an AMQP broker (examples/celery_app.rs),
• the same Celery app implemented in Python (examples/celery_app.py),
• and a Beat app implemented in Rust (examples/beat_app.rs).

Prerequisites

If you already have an AMQP broker running you can set the environment variable AMQP_ADDR to your broker's URL (e.g., amqp://localhost:5672//, where the second slash at the end is the name of the default vhost). Otherwise simply run the helper script:

./scripts/brokers/amqp.sh

This will download and run the official RabbitMQ image (RabbitMQ is a popular AMQP broker).

Run the examples

Run Rust Celery app

You can consume tasks with:

cargo run --example celery_app consume

And you can produce tasks with:

cargo run --example celery_app produce [task_name]

Current supported tasks for this example are: add, buggy_task, long_running_task and bound_task

Run Python Celery app

Similarly, you can consume or produce tasks from Python by running

python examples/celery_app.py consume [task_name]

or

python examples/celery_app.py produce

You'll need to have Python 3 installed, along with the requirements listed in the requirements.txt file. You'll also have to provide a task name. This example implements 4 tasks: add, buggy_task, long_running_task and bound_task

Run Rust Beat app

You can start the Rust beat with:

cargo run --example beat_app

And then you can consume tasks from Rust or Python as explained above.

Redis-backed Beat

A Redis-powered distributed scheduler backend is available through RedisSchedulerBackend. To try it out locally (requires a Redis server running):

REDIS_URL=redis://127.0.0.1:6379/0 cargo run --example redis_beat

Only the instance that holds the Redis lock will dispatch tasks, while followers wait and take over automatically if the leader disconnects.

To test multi-instance failover:

1. Run a worker connected to Redis so scheduled tasks are consumed (see examples/celery_app.rs).
2. Start the first beat instance as shown above; it will log that it acquired leadership.
3. Start a second beat instance with the same command; it stays on standby.
4. Stop the first instance (e.g. Ctrl+C). Within a few seconds the standby will acquire the lock and resume scheduling without losing any tasks.

Road map and current state

✅ = Supported and mostly stable, although there may be a few incomplete features. ⚠️ = Partially implemented and under active development. 🔴 = Not supported yet but on-deck to be implemented soon.

Core

Note: Issue tracking links below reference this repository.

Brokers

Backends

Project History and Maintenance

This is a Community Fork

This project (celery-rs) is a community-maintained fork of the original rusty-celery project. We've taken over maintenance due to the original project becoming inactive.

Key Changes in This Fork:

• ✅ Active Maintenance: Regular updates and bug fixes
• ✅ Updated Dependencies: All dependencies kept up-to-date
• ✅ Improved Stability: Fixed broker connection issues and test reliability
• ✅ Modern Rust: Compatible with latest Rust versions and async ecosystem

Migration from rusty-celery

If you're migrating from the original rusty-celery, the API remains 100% compatible. Simply update your Cargo.toml:

[dependencies]
# Change from:
# celery = "0.5"

# To:
celery-rs = "0.6"
# Or use git directly:
# celery-rs = { git = "https://github.com/GaiaNet-AI/celery-rs", branch = "main" }

Contributing

We welcome contributions! This fork aims to:

• Maintain API compatibility with the original project
• Provide active maintenance and support
• Keep dependencies updated
• Fix bugs and add features requested by the community

Acknowledgments

Special thanks to the original rusty-celery team for creating this excellent foundation. This fork builds upon their work while ensuring continued development and support for the Rust Celery ecosystem.