# Changelog
All notable changes to the `v_queue` crate are recorded in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.3.0]
### Added
- New per-part on-disk format `v3` with optional per-message body compression
using Zstd and a per-part dictionary, to make queues more compact on disk
with little read/write overhead:
- Each `v3` record keeps the same 25-byte header but uses a distinct magic
marker (`MAGIC_MARKER_V3`) and carries a compression flag in the high bit
of the `msg_type` byte. A compressed body is stored as a 4-byte original
length prefix followed by the Zstd payload; the CRC still covers exactly
what is written to disk.
- The dictionary is trained per part during a warmup phase. Early messages
are written uncompressed while a bounded reservoir of samples is collected;
a candidate dictionary is retrained every few hundred messages and its
compression ratio is measured on a recent window. When the ratio stops
improving (saturation) the dictionary is frozen, persisted to
`{queue}_dict` atomically (temp file plus fsync, then rename) before any
record can reference it, and later bodies are compressed against it.
- Consumers detect the part format from `_info_push`, lazily load the part
dictionary on the first compressed record, verify the CRC over the stored
bytes, then decompress. `Consumer::record_len()` exposes the uncompressed
length so callers can size their read buffer.
- New tests `test_v3_compression_roundtrip` (warmup to compressed transition,
byte-for-byte round-trip, and on-disk size reduction) and
`test_v3_blocks_old_format_reader` (v3 info_push and magic marker reject an
old reader).
### Changed
- The writer always creates new parts in `v3` format. When it reuses an empty
previous part it rewrites that part's `_info_push` to `v3` before the first
push.
- `_info_push` of a `v3` part is prefixed with a `v3;` version token. This is
rejected by the `0.2.9` parser, so once `0.3.0` writes a part an old reader
can neither read nor see it. The shared `_info_queue` is left unchanged, so a
`0.2.9` reader can still drain pre-existing `v2` parts.
### Compatibility
- `0.3.0` reads both old `v2` parts and new `v3` parts.
- `0.2.9` cannot read or see `v3` parts. A `0.2.9` consumer already positioned
inside a `v2` part can finish draining it, but a fresh `0.2.9` consumer that
starts at the current `v3` part fails to open it.
## [0.2.9] - 2026-05-06
### Added
- New tests in `src/test.rs` covering the queue reliability fixes plan
(`queue-reliability-tests`):
- `test_push_rollback_on_write_error` — verifies that a `push` whose
underlying write fails (simulated via `/dev/full`) returns `Err`, leaves
`right_edge` and `count_pushed` at the last successfully persisted values,
does not change the on-disk queue file, and that the queue stays usable
after the failure (related to phase 1.2 of the reliability fixes plan).
- `test_queue_new_fails_on_corrupted_info_push_crc` — corrupts the CRC
digit of `<queue>_info_push` and expects `Queue::new(Read)` to return
`Err(InvalidChecksum)` (phase 1.3).
- `test_queue_new_fails_on_corrupted_info_queue_crc` — same idea for the
per-queue `<queue>_info_queue` file (phase 1.3).
- `test_double_open_returns_already_open` — opens a queue in `ReadWrite`
mode, expects a second concurrent `ReadWrite` open to fail with
`AlreadyOpen` while the first is alive, allows a parallel `Read` open,
and re-checks that `ReadWrite` becomes available again after `drop`
(phase 1.1).
- `test_consumer_reconnect_arbitrary_queue_name` — uses a queue whose name
is not the hardcoded `individuals-flow` and verifies that after
`commit + drop + reconnect` the consumer resumes from its last committed
position instead of restarting from zero (phase 2.1).
### Fixed
- Queue and consumer advisory locks are now held for the full lifetime of
`Queue` and `Consumer`, so concurrent `ReadWrite` opens fail with
`AlreadyOpen` instead of racing on the same files.
- `Queue::push` now uses full writes and rolls the queue file back to the last
known good boundary if writing the record or updating push metadata fails.
- Queue metadata files are validated on read: CRC mismatches in
`<queue>_info_queue` and `<queue>_info_push` now fail instead of silently
producing stale or incorrect queue state.
- Consumer body reads now update `pos_record` and `count_popped` only after CRC
verification succeeds, and tail-read failures rewind to the start of the
record for retry.
- Consumer state files now use the actual queue name instead of the hardcoded
`individuals-flow` prefix, so reconnect resumes from the committed position
for arbitrary queue names.
- Info files are truncated before being rewritten, preventing stale bytes from
previous longer records from being parsed by future readers.
- Header validation now checks the magic marker and message bounds before the
body is allocated or read, and corrupted-record recovery requires a full
4-byte marker match.
- Consumers now refresh the current part metadata when checking batch size, so
newly appended messages become visible without waiting for the writer to roll
to a new part.
- Writer restart reuses an empty current part instead of creating another empty
queue part.
### Changed
- Reliability tests from `0.2.9` that were previously documented as ignored are
now active and passing.
## [0.2.8]
- Baseline before the reliability tests were added (no changelog entry kept
for previous releases).