segment-buffer 0.5.3

High-throughput local buffer for cloud sync: batch-spills to zstd+CBOR segment files with at-least-once delivery, ack-based deletion, filename-based crash recovery, configurable durability, and optional encryption. Single-process by design. No WAL, no metadata DB.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164

# Contributing

Thanks for your interest in contributing to segment-buffer!

## How to Contribute

1. Fork the repository
2. Create a feature branch (`git switch -c my-feature`)
3. Make your changes
4. Ensure the full verification suite below passes
5. Submit a pull request

## Development Setup

The `encryption` feature is **off by default**, so most commands must run twice —
once with default features and once with `--features encryption`. CI does exactly
this (see `.github/workflows/ci.yml`).

### Using Cargo directly

```bash
# Tests (canonical command — runs both default + encryption-gated tests)
cargo test --no-fail-fast --features encryption

# Lint (warnings are hard errors, both in CONTRIBUTING and CI via RUSTFLAGS=-D warnings)
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --features encryption -- -D warnings
cargo fmt --all -- --check

# Examples
cargo run --example basic_usage
cargo run --example backpressure
cargo run --example encrypted --features encryption   # REQUIRES the feature flag

# Benchmarks (criterion) — 8 targets declared in Cargo.toml
cargo bench --bench bench_append
cargo bench --bench bench_read_from
cargo bench --bench bench_read_vs_for_each
cargo bench --bench bench_delete_acked
cargo bench --bench bench_recover
cargo bench --bench bench_stats
cargo bench --bench bench_append_all
cargo bench --bench bench_durability_policy

# Docs (build with the feature so AesGcmCipher is visible)
cargo doc --no-deps --features encryption
```

### Using Nix (reproducible)

A `flake.nix` provides a devShell with the pinned Rust toolchain plus the native
`zstd` and `pkg-config` dependencies:

```bash
nix develop          # drop into the devShell
nix flake check      # verify the flake
```

## Code Conventions

- `#![warn(missing_docs)]` is on — every public item needs a doc comment.
- Doc comments use `# Errors` and `# Example` sections where relevant.
- Tests use `tempfile::TempDir`; see `src/tests.rs` for the helper patterns.
- The MSRV is **1.86** (enforced by a dedicated CI job).

## Semver and stability policy

This crate follows [Semantic Versioning](https://semver.org/) with the
following concrete rules:

- **Breaking changes (major bump)**: adding a public item, removing a public
  item, changing a signature, changing on-disk format, changing the `Debug` /
  `Display` output of a public type in a way operators may be parsing, or
  changing behavior in a way that requires downstream action.
- **Additive changes (minor bump)**: adding a new public item, performance
  improvements, doc/test additions, new feature flags, bug fixes that do not
  change documented behavior.
- **Patch-level changes (patch bump)**: regression fixes, doc/test additions
  that don't change the API, dependency bumps within semver.

### `#[non_exhaustive]` is the default for new public structs and enums

Adding a field to a struct or a variant to an enum is otherwise breaking. We
avoid that by marking every new public struct/enum `#[non_exhaustive]` at
birth. `SegmentConfig`, `BufferStats`, and `SegmentError` are all
`#[non_exhaustive]` from v0.3.0 onward. The cost is paid by callers (they
must use `Default + field reassignment` or a builder), not by the API.

### `std::error::Error::source()` chains are required on wrapped errors

When a public error variant wraps a typed underlying error, the underlying
cause must be reachable via `Error::source()` — not erased behind a `format!`.
`CipherError::with_source` is the constructor that honors this; `msg` is
for errors with no underlying cause. Adding a typed underlying error and
_then_ removing it later (or vice versa) is breaking.

### On-disk format changes require a migration story

The on-disk format (filename contract, envelope, CBOR+zstd encoding, cipher
payload format) is part of the API. A change to it is a breaking change AND
requires either an automatic migration path or a documented upgrade
procedure that does not lose data. See the `SBF1` envelope (added in v0.2.0)
for the additive-change pattern: legacy files are auto-detected and read
without migration.

### Internal hooks: `#[cfg]` over `#[doc(hidden)]`

When exposing internals for in-tree fuzz targets or deep integration tests
(the `fuzz_hooks` module pattern), **gate the module behind a Cargo feature
with `#[cfg]`**, never with `#[doc(hidden)] pub`.

Why: `#[doc(hidden)]` hides an item from rustdoc but **does not remove it
from the semver surface.** Once a `pub` item ships in a release, changing
its signature or removing it is technically a breaking change even if no
documentation references it. Downstream users who happen to import the
hidden path will break on the next minor bump.

The `#[cfg(any(test, feature = "fuzz"))]` pattern is the correct shield:

- Without `--features fuzz`, the module does not exist in the compiled
  crate, so downstream users cannot reach the items at all. The semver
  surface stays clean.
- With `--features fuzz`, the items exist; users who explicitly opt in are
  signaling "I accept these are unstable." Document this in the feature
  description in `Cargo.toml`.

The `fuzz` feature on this crate is the canonical example. See `src/lib.rs`
`pub mod fuzz_hooks` and the matching `fuzz` feature in `Cargo.toml`.

### Dependency bumps must respect the MSRV

Before widening any dependency version spec (major or minor bump, including
transitive deps pulled by a `cargo update`), verify the new version's
`rust-version` against this crate's declared MSRV (currently 1.86 — see
`docs/MSRV.md` and the `rust-version` field in `Cargo.toml`).

Why: commit `031763d` bumped `criterion` 0.5 → 0.8 without checking MSRV.
`criterion` 0.8 requires rustc 1.86; CI's MSRV job (then pinned at 1.85) and
the 1.85 matrix jobs all failed, forcing a revert (`c4be692`). This wasted
a full CI cycle on a foreseeable, locally-checkable error. (The MSRV has
since been deliberately bumped to 1.86 to unblock criterion 0.8+, and the
dependabot ignore on `criterion >= 0.6` retired.)

The check:

```bash
# After changing any dep version in Cargo.toml or running cargo update:
cargo +1.86 check --all-targets --features encryption
# Or, more thoroughly, look at the dep tree for rust-version floors:
cargo tree -e normal --duplicates
```

## Reporting Issues

Use [GitHub Issues](https://github.com/LarsArtmann/segment-buffer/issues).

For the current state of the project, see
[FEATURES.md](FEATURES.md) (capability inventory) and
[ROADMAP.md](ROADMAP.md) (direction).

## License

By contributing, you agree that your contributions will be licensed under the
Apache License 2.0.