rust-job-queue-api-worker-system 0.1.0

A production-shaped Rust job queue: Axum API + async workers + Postgres SKIP LOCKED dequeue, retries with decorrelated jitter, idempotency, cooperative cancellation, OpenAPI, Prometheus metrics.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78

//! A production-shaped Rust job queue with a Postgres `SKIP LOCKED` dequeue,
//! retries with decorrelated-jitter backoff, idempotency, cooperative
//! cancellation, OpenAPI documentation, and Prometheus metrics.
//!
//! # Crate shape
//!
//! One Cargo package with two optional feature flags. The crate compiles
//! in any of these configurations:
//!
//! | Configuration | What you get |
//! |---|---|
//! | `--no-default-features` | Core only: domain types, payload validation, queue SQL, retry math, DB pool, migration runner |
//! | `--no-default-features --features api` | Core + Axum router, OpenAPI/Swagger UI, Prometheus exporter |
//! | `--no-default-features --features worker` | Core + Tokio worker pool, `Executor` trait, deterministic simulator, graceful shutdown |
//! | _(default features)_ | All of the above |
//!
//! Two binaries plus a migration runner:
//!
//! | Binary | Feature required | Purpose |
//! |---|---|---|
//! | `job-queue-api` | `api` | HTTP server |
//! | `job-queue-worker` | `worker` | Async job processor |
//! | `job-queue-migrate` | _(none)_ | One-shot migration runner |
//!
//! # Minimal example
//!
//! Enqueue a job from an embedding application using core only:
//!
//! ```no_run
//! use rust_job_queue_api_worker_system::{
//!     connect, migrate, JobKind, NewJob, PoolConfig,
//!     queue::enqueue,
//! };
//! use serde_json::json;
//!
//! # async fn run() -> anyhow::Result<()> {
//! let pool = connect(&PoolConfig::from_url("postgres://localhost/jobs")).await?;
//! migrate(&pool).await?;
//!
//! let outcome = enqueue(&pool, NewJob {
//!     kind: JobKind::SendEmail,
//!     payload: json!({ "to": "a@b.c", "subject": "hi", "body": "hello" }),
//!     max_attempts: Some(3),
//!     idempotency_key: Some("user-42-welcome".into()),
//! }).await?;
//! println!("queued {}", outcome.job().id);
//! # Ok(()) }
//! ```
//!
//! # Where to go next
//!
//! - [`queue`] — the dequeue / mark / cancel operations. The SKIP LOCKED
//!   statement is in [`queue::fetch_next`].
//! - [`worker`] — the `Executor` trait and `WorkerRuntime` (feature `worker`).
//! - [`api`] — the Axum router and `AppState` (feature `api`).
//! - [`docs/`](https://github.com/infinityabundance/Rust-Job-Queue-API-Worker-System/tree/main/docs)
//!   in the repo for `architecture.md`, `api.md`, `tradeoffs.md`, `runbook.md`.

#![forbid(unsafe_code)]

pub mod db;
pub mod domain;
pub mod error;
pub mod ids;
pub mod payload;
pub mod queue;
pub mod retry;

#[cfg(feature = "api")]
pub mod api;

#[cfg(feature = "worker")]
pub mod worker;

pub use db::{connect, migrate, PoolConfig};
pub use domain::{Job, JobKind, JobOutcome, JobStatus, NewJob};
pub use error::JobError;
pub use ids::JobId;