Enum QueueDriver 

pub enum QueueDriver {
    Sync,
    Memory,
    Database,
    Redis,
    Sqs,
    Kafka,
}

Which Queue to build.

use rainier_queue::QueueDriver;
use rainier_support::Setting;

assert!(!QueueDriver::Sync.is_deferred(), "sync runs the job inline");
assert!(QueueDriver::parse("database").unwrap().survives_a_restart());

Variants

Sync

Run the job inline, on the thread that dispatched it.

The default because it needs no worker: a fresh clone dispatches a job and the job happens. Also the reason a slow job makes a slow request — which is the thing a queue exists to avoid, so this is a development setting.

Memory

A queue in this process’s memory.

For tests that want a job to be queued rather than run, so they can assert on it. Lost on restart, invisible to other processes.

Database

Two tables in the application’s own database.

Durable, shared, and needs no new infrastructure — the reason it is the usual production answer. Needs the migrations (DatabaseQueue::migrations()) and a queue:work process.

Redis

Redis streams.

Durable-ish, and the qualifier is the point: Redis acknowledges a write before it is on disk and before a replica has it, so a job can be lost to a crash or a failover. Right for work you can afford to lose, wrong for work you cannot — see the driver’s docs. Needs the redis feature.

Sqs

Amazon SQS.

Durable and managed, with the visibility timeout doing the reservation. Needs the sqs feature.

Kafka

A Kafka topic.

Read the driver’s docs before choosing it. A log is not a queue: concurrency is the partition count rather than the worker count, a delayed job blocks its partition, and a retry goes to the end of the topic instead of back where it was. Right when the jobs are already events on a topic; wrong as a general job queue. Needs the kafka feature.

Implementations

impl QueueDriver

pub fn as_str(&self) -> &'static str

The wire spelling — what appears in .env and in the config tree.

Inherent as well as on the trait, so callers do not have to import Setting to print one.

impl QueueDriver

pub fn is_deferred(&self) -> bool

Whether dispatching returns before the job has run.

false for Sync, which is the whole difference between it and the others — and the reason a test that asserts on a side effect passes under sync and hangs under everything else.

pub fn survives_a_restart(&self) -> bool

Whether a job dispatched now survives this process exiting.

True for anything that stores the job somewhere else. It is not the same question as whether the job is safe — see may_lose_an_accepted_job, which is the one that catches Redis.

pub fn may_lose_an_accepted_job(&self) -> bool

Whether a dispatch this driver accepted can still be lost.

True only for Redis, and it is the distinction the obvious question misses: a Redis job survives your process restarting, because it lives in another one — and can still vanish, because Redis acknowledges a write before it is on disk (appendfsync everysec leaves a window of about a second) and before any replica has it ( replication is asynchronous, so a failover can drop confirmed writes).

A database queue’s insert is committed before it returns; SQS is replicated across availability zones before it acknowledges. Neither can tell you a job is queued and then lose it.

Kafka is produced to with acks=all, so a push returns once every in-sync replica holds the record — which is the same guarantee, and is only as good as the replication. A single-broker cluster has one in-sync replica and Kafka does not fsync each write, so a development cluster can lose an accepted job even though this reports it cannot. replication.factor and min.insync.replicas of at least two are what make the answer true.

So this is the predicate a deploy check should look at when the work is a payment rather than a cache warm.

pub fn needs_a_worker(&self) -> bool

Whether this driver needs a queue:work process to make progress.

Trait Implementations

impl AsRef<str> for QueueDriver

fn as_ref(&self) -> &str

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for QueueDriver

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QueueDriver

impl Debug for QueueDriver

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for QueueDriver

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for QueueDriver

fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for QueueDriver

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for QueueDriver

impl FromStr for QueueDriver

type Err = Error

The associated error which can be returned from parsing.

fn from_str(raw: &str) -> Result<Self, Self::Err>

Parses a string s to return a value of this type.

impl Hash for QueueDriver

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for QueueDriver

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for QueueDriver

fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.

impl Setting for QueueDriver

const SETTING: &'static str = "queue driver"

What this setting is called in an error message — "cache driver".

const ALL: &'static [Self]

Every variant, in the order they should be listed to a human.

fn as_str(&self) -> &'static str

The wire spelling — what appears in .env, in the config tree, and in serialised output.

fn parse(raw: &str) -> Result<Self, Error>

Parse a wire spelling.

fn options() -> String

The valid spellings, backtick-quoted and comma-separated.

impl StructuralPartialEq for QueueDriver