v_queue 0.3.0

# Changelog

All notable changes to the `v_queue` crate are recorded in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.3.0]

### Added

- New per-part on-disk format `v3` with optional per-message body compression
  using Zstd and a per-part dictionary, to make queues more compact on disk
  with little read/write overhead:
  - Each `v3` record keeps the same 25-byte header but uses a distinct magic
    marker (`MAGIC_MARKER_V3`) and carries a compression flag in the high bit
    of the `msg_type` byte. A compressed body is stored as a 4-byte original
    length prefix followed by the Zstd payload; the CRC still covers exactly
    what is written to disk.
  - The dictionary is trained per part during a warmup phase. Early messages
    are written uncompressed while a bounded reservoir of samples is collected;
    a candidate dictionary is retrained every few hundred messages and its
    compression ratio is measured on a recent window. When the ratio stops
    improving (saturation) the dictionary is frozen, persisted to
    `{queue}_dict` atomically (temp file plus fsync, then rename) before any
    record can reference it, and later bodies are compressed against it.
  - Consumers detect the part format from `_info_push`, lazily load the part
    dictionary on the first compressed record, verify the CRC over the stored
    bytes, then decompress. `Consumer::record_len()` exposes the uncompressed
    length so callers can size their read buffer.
- New tests `test_v3_compression_roundtrip` (warmup to compressed transition,
  byte-for-byte round-trip, and on-disk size reduction) and
  `test_v3_blocks_old_format_reader` (v3 info_push and magic marker reject an
  old reader).

### Changed

- The writer always creates new parts in `v3` format. When it reuses an empty
  previous part it rewrites that part's `_info_push` to `v3` before the first
  push.
- `_info_push` of a `v3` part is prefixed with a `v3;` version token. This is
  rejected by the `0.2.9` parser, so once `0.3.0` writes a part an old reader
  can neither read nor see it. The shared `_info_queue` is left unchanged, so a
  `0.2.9` reader can still drain pre-existing `v2` parts.

### Compatibility

- `0.3.0` reads both old `v2` parts and new `v3` parts.
- `0.2.9` cannot read or see `v3` parts. A `0.2.9` consumer already positioned
  inside a `v2` part can finish draining it, but a fresh `0.2.9` consumer that
  starts at the current `v3` part fails to open it.

## [0.2.9] - 2026-05-06

### Added

- New tests in `src/test.rs` covering the queue reliability fixes plan
  (`queue-reliability-tests`):
  - `test_push_rollback_on_write_error` — verifies that a `push` whose
    underlying write fails (simulated via `/dev/full`) returns `Err`, leaves
    `right_edge` and `count_pushed` at the last successfully persisted values,
    does not change the on-disk queue file, and that the queue stays usable
    after the failure (related to phase 1.2 of the reliability fixes plan).
  - `test_queue_new_fails_on_corrupted_info_push_crc` — corrupts the CRC
    digit of `<queue>_info_push` and expects `Queue::new(Read)` to return
    `Err(InvalidChecksum)` (phase 1.3).
  - `test_queue_new_fails_on_corrupted_info_queue_crc` — same idea for the
    per-queue `<queue>_info_queue` file (phase 1.3).
  - `test_double_open_returns_already_open` — opens a queue in `ReadWrite`
    mode, expects a second concurrent `ReadWrite` open to fail with
    `AlreadyOpen` while the first is alive, allows a parallel `Read` open,
    and re-checks that `ReadWrite` becomes available again after `drop`
    (phase 1.1).
  - `test_consumer_reconnect_arbitrary_queue_name` — uses a queue whose name
    is not the hardcoded `individuals-flow` and verifies that after
    `commit + drop + reconnect` the consumer resumes from its last committed
    position instead of restarting from zero (phase 2.1).

### Fixed

- Queue and consumer advisory locks are now held for the full lifetime of
  `Queue` and `Consumer`, so concurrent `ReadWrite` opens fail with
  `AlreadyOpen` instead of racing on the same files.
- `Queue::push` now uses full writes and rolls the queue file back to the last
  known good boundary if writing the record or updating push metadata fails.
- Queue metadata files are validated on read: CRC mismatches in
  `<queue>_info_queue` and `<queue>_info_push` now fail instead of silently
  producing stale or incorrect queue state.
- Consumer body reads now update `pos_record` and `count_popped` only after CRC
  verification succeeds, and tail-read failures rewind to the start of the
  record for retry.
- Consumer state files now use the actual queue name instead of the hardcoded
  `individuals-flow` prefix, so reconnect resumes from the committed position
  for arbitrary queue names.
- Info files are truncated before being rewritten, preventing stale bytes from
  previous longer records from being parsed by future readers.
- Header validation now checks the magic marker and message bounds before the
  body is allocated or read, and corrupted-record recovery requires a full
  4-byte marker match.
- Consumers now refresh the current part metadata when checking batch size, so
  newly appended messages become visible without waiting for the writer to roll
  to a new part.
- Writer restart reuses an empty current part instead of creating another empty
  queue part.

### Changed

- Reliability tests from `0.2.9` that were previously documented as ignored are
  now active and passing.

## [0.2.8]

- Baseline before the reliability tests were added (no changelog entry kept
  for previous releases).