Event Queues

In-memory and RocksDB-backed event queues for Arch services.

arch-event-queues = "0.1"

The package name is arch-event-queues; the Rust import path is arch_event_queues.

In-Memory Queue

Use EventQueue<T> when events only need to live in memory.

use arch_event_queues::EventQueue;

#[derive(Debug, PartialEq, Eq)]
struct Event {
    payload: String,
}

let queue = EventQueue::new();
queue
    .push(Event {
        payload: "created".to_string(),
    })
    .unwrap();

assert_eq!(
    queue.pop().unwrap(),
    Some(Event {
        payload: "created".to_string(),
    })
);

Durable Queue

Use DurableEventQueue<T> when events must survive process restarts until they are acknowledged. Payloads must implement borsh::BorshSerialize, borsh::BorshDeserialize, and Clone.

use arch_event_queues::DurableEventQueue;
use borsh::{BorshDeserialize, BorshSerialize};

#[derive(Clone, Debug, PartialEq, Eq, BorshSerialize, BorshDeserialize)]
struct Job {
    payload: String,
}

let dir = tempfile::tempdir().unwrap();
let queue = DurableEventQueue::open(dir.path()).unwrap();

let queued = queue
    .push(Job {
        payload: "index block".to_string(),
    })
    .unwrap();

let next = queue.pop().unwrap().expect("queued job");
assert_eq!(next.id, queued.id);

queue.ack(next.id).unwrap();

Eager vs Lazy

DurableEventQueue::open(path) uses eager mode. On startup, it scans RocksDB, deserializes every unacked event, and fills the in-memory ready queue. This makes polling cheap after open, but startup cost grows with the number and size of unacked events.

DurableEventQueueOptions::lazy() uses lazy mode. Startup scans keys to find ids but does not deserialize event payloads until they are popped. This is better for large backlogs or large payloads.

use arch_event_queues::{DurableEventQueue, DurableEventQueueOptions};

#[derive(Clone, borsh::BorshSerialize, borsh::BorshDeserialize)]
struct Job;

let dir = tempfile::tempdir().unwrap();
let queue =
    DurableEventQueue::<Job>::open_with_options(dir.path(), DurableEventQueueOptions::lazy())
        .unwrap();