Skip to main content

Module elastic_queue

Module elastic_queue 

Source
Expand description

Rust port of the Java ElasticQueue + FileElasticStore (org.platformlambda.core.util) — the reactive back-pressure overflow buffer behind every route’s manager (ServiceQueue analog).

A per-route two-tier FIFO: the first MEMORY_BUFFER events of a burst stay in memory, then the overflow spills to fixed-size append-only segment files. Record format is portable and kept byte-identical to the Java file store: [4-byte big-endian length][payload]. A segment is sealed once it passes the size threshold (elastic.queue.segment.size.bytes, default 16 MB, min 512); a sealed, fully-consumed segment is deleted immediately — O(1) reclamation, no compaction, no cleaner thread (the whole point vs. the legacy Berkeley DB store, which is not ported — maintainer decision 2026-07-15; with one store the Java ElasticStore strategy facade collapses into this single type).

Threading: single-threaded per route (driven by the route’s manager task). The buffer is transient — not durable across restart; there is no fsync. Base directory: transient.data.store (default /tmp/reactive, kept verbatim — D9) + <application name>-<process origin> (Platform identity) unless running.in.cloud=true.

Housekeeping (Java ensureBaseDir + periodic keep-alive + shutdown hook, wired by the lifecycle): start_housekeeping refreshes the RUNNING liveness marker every 20 s and runs scan_expired_stores once (removes sibling holding areas whose marker is stale for > 1 h, or unknown dirs holding segment files); shutdown_cleanup purges segments + the marker on graceful exit (Java uses a JVM shutdown hook; signal handling is a later increment — call it from your main).

Structs§

ElasticQueue
The per-route elastic overflow buffer. Reserved for system use — the route manager drives it; do not use directly in application code (Java carries the same warning on ServiceQueue).

Constants§

MEMORY_BUFFER
First this many events of a burst are held in memory before spilling to disk (Java ElasticStore.MEMORY_BUFFER).

Functions§

base_dir
The holding area for segment files, created once per process (Java FileElasticStore.ensureBaseDir).
scan_expired_stores
Remove sibling holding areas left behind by dead processes (Java scanExpiredStores/removeExpiredStore): a dir whose RUNNING marker went stale (> 1 h) while still holding segment files has expired; a dir holding segment files with no marker is unknown debris. Both are removed.
shutdown_cleanup
Graceful-exit cleanup (Java’s JVM shutdown hook): purge this process’s segment files and remove the RUNNING marker. Call from your main after the lifecycle completes; OS-signal wiring is a later increment.
start_housekeeping
Start the housekeeping the lifecycle owns (idempotent — Java wires this in ensureBaseDir + a Vert.x periodic timer): refresh the RUNNING marker every 20 s so siblings can tell this holding area is alive, and scan once for expired/unknown holding areas. Must be called within a Tokio runtime (AppStarter::run does — its “essential services” phase).