Expand description
An append-only log for storing fixed length items on disk.
In addition to replay, stored items can be fetched directly by their position in the journal,
where position is defined as the item’s order of insertion starting from 0, unaffected by
pruning.
See crate::journal::variable for a journal that supports variable length items.
§Format
Data stored in a fixed::Journal is persisted in one of many Blobs within a caller-provided
partition. Each Blob contains a configurable maximum of items_per_blob, with each item
followed by its checksum (CRC32):
+--------+-----------+--------+-----------+--------+----------+-------------+
| item_0 | C(Item_0) | item_1 | C(Item_1) | ... | item_n-1 | C(Item_n-1) |
+--------+-----------+--------+----0------+--------+----------+-------------+
n = config.items_per_blob, C = CRC32The most recent blob may not necessarily be full, in which case it will contain fewer than the maximum number of items.
A fetched or replayed item’s checksum is always computed and checked against the stored value before it is returned. If the checksums do not match, an error is returned instead.
§Open Blobs
All Blobs in a given partition are kept open during the lifetime of Journal. You can limit
the number of open blobs by using a higher number of items_per_blob or pruning old items.
§Consistency
Data written to Journal may not be immediately persisted to Storage. It is up to the caller
to determine when to force pending data to be written to Storage using the sync method. When
calling close, all pending data is automatically synced and any open blobs are closed.
§Pruning
The prune method allows the Journal to prune blobs consisting entirely of items prior to a
given point in history.
§State Sync
Journal::init_sync allows for initializing a journal for use in state sync.
When opened in this mode, we attempt to populate the journal within the given range
with persisted data.
If the journal is empty, we create a fresh journal at the specified position.
If the journal is not empty, we prune the journal to the specified lower bound and rewind to
the specified upper bound.
§Replay
The replay method supports fast reading of all unpruned items into memory.