brec 0.4.0

`brec` is a tool that allows you to quickly and easily create a custom message exchange protocol with resilience to data "corruption" and the ability to extract messages from mixed streams (i.e., streams containing not only `brec` packets but also any other data). `brec` is developed for designing your own custom binary protocol - without predefined message formats or rigid schemas.

> **Notice**: Public Beta
>
> The `brec` is currently in a public beta phase. Its core functionality has demonstrated strong reliability under heavy stress testing, and the system is considered stable for most use cases.
>
> However, the public API is still evolving as we work to support a wider range of scenarios.
> We welcome your feedback and would be grateful if you share which features or improvements would make brec more valuable for your needs.
>
> **Thanks for being part of the journey!**

# Key Features

- **Protocol without constraints** - Unlike many alternatives, `brec` doesn’t enforce a fixed message layout. Instead, you define your own building blocks (`blocks`) and arbitrary payloads (`payloads`), combining them freely into custom packets.
- **Stream-recognizable messages** - Each block, payload, and packet is automatically assigned a unique signature, making them easily discoverable within any byte stream.
- **Built-in reliability** - All parts of a packet (blocks, payloads, and headers) are automatically linked with their own CRC checksums to ensure data integrity.
- **Stream-aware reading** - `brec` includes a powerful streaming reader capable of extracting packets even from noisy or corrupted streams - skipping irrelevant or damaged data without breaking.
- **Non-packet data is preserved** - When reading mixed streams, unrecognized data is not lost. You can capture and process it separately using rules and callbacks.
- **Persistent storage layer** - `brec` provides a high-performance storage engine for persisting packets. Its slot-based layout enables fast indexed access, filtering, and direct access by packet index.
- **Payload runtime context** - Payloads may depend on explicit runtime state during encoding and decoding.
- **Optional payload encryption** - With the `crypt` feature, selected payloads can be encrypted while others in the same protocol remain open.
- **Optional resilient compatibility** - With the `resilient` feature, older readers can skip unknown blocks and payloads during protocol evolution.
- **Optional Node.js bridge (N-API)** - With the `napi` feature, protocol objects can be converted directly between Rust and JavaScript without JSON conversion as an intermediate transport.
- **Optional WASM bridge (`wasm-bindgen`)** - With the `wasm` feature, protocol objects can be converted directly between Rust and JavaScript in browser/wasm runtimes without JSON as an intermediate transport.
- **Optional Java bridge (JNI)** - With the `java` feature, protocol objects can be converted directly between Rust and Java runtime objects without JSON as an intermediate transport.
- **Optional C# bridge (PInvoke / C ABI)** - With the `csharp` feature, protocol objects can be converted directly between Rust packet models and a stable Rust-side value ABI for .NET-facing integrations.
- **High performance** - Parsing performance is on par with the most optimized binary parsers (see the Performance section).
- **Simple to use** - Just annotate your structs with #[block] or #[payload], and brec takes care of the rest - your protocol is ready to go.

# General Overview

The primary unit of information in `brec` is a packet (`Packet`) - a ready-to-transmit message with a unique signature (allowing it to be recognized within mixed data) and a CRC to ensure data integrity.

A packet consists of a set of blocks (`Block`) and, optionally, a payload (`Payload`).

![Scheme](./assets/scheme.svg)

Blocks (`Block`) are the minimal units of information in the `brec` system. A block can contain only primitives, such as numbers, boolean values, and byte slices. A block serves as a kind of packet index, allowing for quick determination of whether a packet requires full processing (i.e., parsing the `Payload`) or can be ignored.

The payload (`Payload`) is an optional part of the packet. Unlike blocks (`Block`), it has no restrictions on the type of data it can contain-it can be a `struct` or `enum` of any complexity and nesting level.

Unlike most protocols, `brec` does not require users to define a fixed set of messages but does require them to describe blocks (`Block`) and payload data (`Payload`).

Users can construct packets (messages) by combining various sets of blocks and payloads. This means `brec` does not impose a predefined list of packets (`Packet`) within the protocol but allows them to be defined dynamically. As a result, the same block and/or payload can be used across multiple packets (messages) without any restrictions.

# Features

## Simplicity of Protocol Type Definition
`brec` includes powerful macros that allow defining the components of a protocol with minimal effort. For example, to define a structure as a block (`Block`), you simply need to use the `block` macro:

```ignore
#[brec::block]
pub struct MyBlock {
    pub field_u8: u8,
    pub field_u16: u16,
    pub field_u32: u32,
    pub field_u64: u64,
    pub field_u128: u128,
    pub field_i8: i8,
    pub field_i16: i16,
    pub field_i32: i32,
    pub field_i64: i64,
    pub field_i128: i128,
    pub field_f32: f32,
    pub field_f64: f64,
    pub field_bool: bool,
    pub blob_a: [u8; 1],
    pub blob_b: [u8; 100],
    pub blob_c: [u8; 1000],
    pub blob_d: [u8; 10000],
}
```

The `block` macro automatically generates all the necessary code for `MyBlock` to function as a block. Specifically, it adds:
- A unique block signature based on its name and path.
- A `CRC` field to ensure data integrity.

All user-defined blocks are ultimately included in a generated enumeration `Block`, as shown in the example:

```ignore
#[brec::block]
pub struct MyBlockA {
    pub field: u8,
    pub blob: [u8; 100],
}

#[brec::block]
pub struct MyBlockB {
    pub field_u16: u16,
    pub field_u32: u32,
}

#[brec::block]
pub struct MyBlockC {
    pub field: u64,
    pub blob: [u8; 10000],
}

// Instruct `brec` to generate and include all protocol types
brec::generate!();

// Generated by `brec`
pub enum Block {
    MyBlockA(MyBlockA),
    MyBlockB(MyBlockB),
    MyBlockC(MyBlockC),
}
```

The generated `Block` enumeration is always returned to the user as a result of message (packet) parsing, allowing for easy identification of blocks by their names.

Similarly to blocks, defining a payload can be done with a simple call to the `payload` macro.

```ignore
#[derive(serde::Deserialize, serde::Serialize)]
pub enum MyNestedEntity {
    One(String),
    Two(Vec<u8>),
    Three,
}

#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub struct MyPayload {
    pub field_u8: u8,
    pub field_u16: u16,
    pub field_u32: u32,
    pub field_u64: u64,
    pub field_u128: u128,
    pub field_nested: MyNestedEntity,
}
```

A payload is an unrestricted set of data (unlike `Block`, which is limited to primitive data). It can be either a `struct` or an `enum` with unlimited nesting levels. Additionally, there are no restrictions on the use of generic types.

`brec` imposes only one requirement for payloads: they must implement the traits `PayloadEncode` and `PayloadDecode<T>`, which enable encoding and decoding of data into target types.

Out of the box (with the `bincode` feature), `brec` provides automatic support for the required traits by leveraging the `bincode` crate. This means that to define a fully functional payload, it is enough to use `#[payload(bincode)]`, eliminating the need for manually implementing the `PayloadEncode` and `PayloadDecode<T>` traits. 

Note that `bincode` requires serialization and deserialization support, which is why the previous examples include `#[derive(serde::Deserialize, serde::Serialize)]`.

If your payload needs runtime state during encoding or decoding, see the `Payload Context` section below.

If your payload should also be encrypted transparently, the recommended path is `#[payload(bincode, crypt)]` with the `crypt` feature enabled.

With `brec`, defining protocol types (blocks and payloads) is reduced to simply defining structures and annotating them with the `block` and `payload` macros.

## Workspace Layout

The repository is organized by responsibility:

- `lib/core` - the public `brec` crate and the protocol/runtime API
- `lib/consts` - shared wire-format constants
- `generator/*` - parsing and proc-macro code generation
- `integration/*` - language-specific runtime bridges and generators
- `tests/*` - CI-oriented test suites covering the core functionality; the same areas also have stress runs where the total generated data volume can reach roughly 40 GB
- `scripts/*` - helper scripts for coverage collection, reporting, and other repository maintenance tasks
- `examples/*` - real usage examples for common `brec` scenarios
- `e2e/*` - end-to-end examples of real integration with other languages and runtimes such as Node.js, WASM, Java, and C#
- `site/*` - project documentation source
- `measurements/*` - performance evaluation and comparison against major alternatives

This keeps the public crate small while allowing integration-specific logic to evolve independently.

## C# (Rust <-> C#)

When the `csharp` feature is enabled, `brec` adds direct Rust <-> C# conversion for generated protocol types through a Rust-side value ABI.

This is useful for PInvoke / C ABI integrations where you want to avoid extra conversion layers like:

1. Rust values -> JSON string
2. JSON string -> C# values

and the reverse path on encode.

### Enabling

```toml
[dependencies]
brec = { version = "...", features = ["csharp", "bincode"] }
```

### Nested Payload Types

For payload C# conversion, nested custom Rust types should derive `brec::CSharp`:

```ignore
#[derive(serde::Serialize, serde::Deserialize, brec::CSharp)]
pub struct MyNestedType { ... }
```

### Conversion Contract

- generated helpers exchange `brec::csharp_feat::CSharpValue`
- float values preserve exact IEEE-754 bit patterns
- field names and variant names are preserved exactly from the Rust protocol model

For complete usage, reflection rules, and packet/object shapes in C#, see:
https://icsmw.github.io/brec/integrations/csharp/

## NAPI (Rust <-> JS)

When the `napi` feature is enabled, `brec` adds direct Rust <-> JavaScript conversion for generated protocol types.

This is primarily useful for Node.js integrations where you want to avoid extra conversion layers like:

1. Rust values -> JSON string
2. JSON string -> JS values

and the reverse path on encode.

With N-API conversion, protocol values are mapped directly between Rust and JS.

### Enabling

```toml
[dependencies]
brec = { version = "...", features = ["napi", "bincode"] }
```

### Nested Payload Types

For payload NAPI conversion, nested custom Rust types should derive `brec::Napi`:

```ignore
#[derive(serde::Serialize, serde::Deserialize, brec::Napi)]
pub struct MyNestedType { ... }
```

### Numeric Mapping

- `i64`, `u64`, `i128`, `u128` are represented via JS `BigInt`
- `f64` is represented via JS `BigInt` as a lossless bit pattern

For complete usage, reflection rules, and packet/object shapes in JS, see:
https://icsmw.github.io/brec/integrations/napi/

## WASM (Rust <-> JS)

When the `wasm` feature is enabled, `brec` adds direct Rust <-> JavaScript conversion for generated protocol types in `wasm-bindgen` environments.

This is useful for browser/wasm integrations where you want to avoid extra conversion layers like:

1. Rust values -> JSON string
2. JSON string -> JS values

and the reverse path on encode.

### Enabling

```toml
[dependencies]
brec = { version = "...", features = ["wasm", "bincode"] }
```

### Nested Payload Types

For payload WASM conversion, nested custom Rust types should derive `brec::Wasm`:

```ignore
#[derive(serde::Serialize, serde::Deserialize, brec::Wasm)]
pub struct MyNestedType { ... }
```

### Numeric Mapping

- `i64`, `u64`, `i128`, `u128` are represented via JS `BigInt`
- `f64` is represented via JS `BigInt` as a lossless bit pattern

For complete usage, reflection rules, and packet/object shapes in JS, see:
https://icsmw.github.io/brec/integrations/wasm/

## Java (Rust <-> Java)

When the `java` feature is enabled, `brec` adds direct Rust <-> Java conversion for generated protocol types via JNI.

This is useful for Java integrations where you want to avoid extra conversion layers like:

1. Rust values -> JSON string
2. JSON string -> Java values

and the reverse path on encode.

### Enabling

```toml
[dependencies]
brec = { version = "...", features = ["java", "bincode"] }
```

### Nested Payload Types

For payload Java conversion, nested custom Rust types should derive `brec::Java`:

```ignore
#[derive(serde::Serialize, serde::Deserialize, brec::Java)]
pub struct MyNestedType { ... }
```

### Numeric Mapping

- `u8`, `u16`, `u32`, `i8`, `i16`, `i32` are mapped through `java.lang.Long` with range checks
- `i64`, `u64`, `i128`, `u128` are mapped via `java.math.BigInteger`

For complete usage, reflection rules, and packet/object shapes in Java, see:
https://icsmw.github.io/brec/integrations/java/

## Simple Packet Construction
Once the protocol data types have been defined, the next step is to include the "unifying" code generated by `brec` using `brec::generate!();`

```ignore
#[brec::block]
pub struct MyBlockA { ... }

#[brec::block]
pub struct MyBlockB { ... }

#[brec::block]
pub struct MyBlockC { ... }

#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub struct MyPayloadA { ... }

#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub struct MyPayloadB { ... }

#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub struct MyPayloadC { ... }

// Instruct `brec` to generate and include all protocol types
brec::generate!();

// Now available:

// Generalized block representation
pub enum Block {
    MyBlockA(MyBlockA),
    MyBlockB(MyBlockB),
    MyBlockC(MyBlockC),
}

// Generalized payload representation
pub enum Payload {
    MyPayloadA(MyPayloadA),
    MyPayloadB(MyPayloadB),
    MyPayloadC(MyPayloadC),
}

// Packet type
pub type Packet = brec::PacketDef<Block, Payload, Payload>;
```

Once all protocol types are defined and the unifying code is generated, you can start creating packets:

```ignore
let my_packet = Packet::new(
    // You are limited to 255 blocks per packet.
    vec![
        Block::MyBlockA(MyBlockA::default()),
        Block::MyBlockC(MyBlockC::default()),
    ],
    // Note: payload is optional
    Some(Payload::MyPayloadA(MyPayloadA::default()))
);
```

At this point, your protocol is ready for use. `Packet` implements all the necessary methods for reading from and writing to a data source.

## Performance, Security, and Efficiency
`brec` is a binary protocol, meaning data is always transmitted and stored in a binary format.

The protocol ensures security through the following mechanisms:
- Each block includes a unique signature generated based on the block's name. Name conflicts within a single crate are eliminated, as the module path is taken into account.
- Similar to blocks, each payload also has a unique signature derived from its name.
- Additionally, `Packet` itself has a fixed 64-bit signature.

These features enable reliable entity recognition within a data stream. Furthermore, blocks, payloads, and the packet itself have their own CRCs. While blocks always use a 32-bit CRC, payloads allow for optional support of 64-bit or 128-bit CRC to enhance protocol security.

`brec` ensures maximum performance through the following optimizations:
- Minimization of data copying and cloning operations.
- Incremental packet parsing: first, blocks are parsed, allowing the user to inspect them and decide whether the packet should be fully parsed (including the payload) or skipped. This enables efficient packet filtering based on block values, avoiding the overhead of parsing a heavy payload.
- If data integrity verification is not required, `brec` allows CRC to be disabled for all types or selectively. This improves performance by eliminating the need for hash calculations.

The conceptual separation of a packet into blocks and a payload allows users to efficiently manage traffic load. Blocks can carry "fast" information that requires quick access, while the payload can implement more complex encoding/decoding mechanisms (such as data compression). Filtering based on blocks helps avoid unnecessary operations on the payload when they are not required.

# Protocol Definition

## Blocks

Any structure can be used as a block, provided it contains fields of the following types:

| Type   |
|--------|
| u8     |
| u16    |
| u32    |
| u64    |
| u128   |
| i8     |
| i16    |
| i32    |
| i64    |
| i128   |
| f32    |
| f64    |
| bool   |
| [u8; n] |

A block can also include an `enum`, but only if it can be converted into one of the supported types. For example:

```ignore
pub enum Level {
    Err,
    Warn,
    Info,
    Debug,
}

// Ensure conversion of `Level` to `u8`
impl From<&Level> for u8 {
    fn from(value: &Level) -> Self {
        match value {
            Level::Err => 0,
            Level::Warn => 1,
            Level::Debug => 2,
            Level::Info => 3,
        }
    }
}

// Ensure conversion of `u8` to `Level`
impl TryFrom<u8> for Level {
    type Error = String;
    fn try_from(value: u8) -> Result<Self, Self::Error> {
        match value {
            0 => Ok(Level::Err),
            1 => Ok(Level::Warn),
            2 => Ok(Level::Debug),
            3 => Ok(Level::Info),
            invalid => Err(format!("{invalid} isn't a valid value for Level")),
        }
    }
}

#[block]
pub struct BlockWithEnum {
    pub level: Level,
    pub data: [u8; 200],
}
```

A structure is declared as a block using the `block` macro. If the block is located outside the visibility scope of the `brec::generate!()` code generator call, the module path must be specified using the `path` directive:

```ignore
#[block(path = mod_a::mod_b)]
pub struct BlockWithEnum {
    pub level: Level,
    pub data: [u8; 200],
}

// Since the `path` directive is the default, the path can also be defined directly:

#[block(mod_a::mod_b)]
pub struct BlockWithEnum {
    pub level: Level,
    pub data: [u8; 200],
}
```

However, in most cases, this approach is not recommended. It is better to ensure the visibility of all blocks at the location where the `brec::generate!()` code generator is invoked.

The code generator provides support for the following traits:

| Trait                  | Available Methods | Return Type | Purpose |
|------------------------|------------------|-------------|---------|
| `ReadBlockFrom`       | `read<T: std::io::Read>(buf: &mut T, skip_sig: bool)` | `Result<Self, Error>` | Attempts to read a block with an option to skip signature recognition (e.g., if the signature has already been read and the user knows exactly which block is being parsed). Returns an error if reading fails for any reason. |
| `ReadBlockFromSlice`  | `read_from_slice<'b>(src: &'b [u8], skip_sig: bool)` | `Result<Self, Error>` | Attempts to read a block from a byte slice, with an option to skip signature verification. Unlike other methods, this returns a reference to a block representation generated by `brec`, rather than the user-defined block itself (see details below). |
| `TryReadFrom`         | `try_read<T: std::io::Read + std::io::Seek>(buf: &mut T)` | `Result<ReadStatus<Self>, Error>` | Attempts to read a block, but instead of returning an error when data is insufficient, it returns a corresponding read status. Also, it moves the source's position only upon successful reading, otherwise keeping it unchanged. |
| `TryReadFromBuffered` | `try_read<T: std::io::BufRead>(reader: &mut T)` | `Result<ReadStatus<Self>, Error>` | Identical to `TryReadFrom`, except it returns a reference to the generated block representation (see details below). |
| `WriteTo`             | `write<T: std::io::Write>(&self, writer: &mut T)` | `std::io::Result<usize>` | Equivalent to the standard `write` method, returning the number of bytes written to the output. Does not guarantee flushing to the output, so calling `flush` is required if such guarantees are needed. |
| `WriteTo`             | `write_all<T: std::io::Write>(&self, writer: &mut T)` | `std::io::Result<()>` | Equivalent to the standard `write_all` method. |
| `WriteVectoredTo`     | `slices(&self)` | `std::io::Result<brec::IoSlices>` | Returns the binary representation of the block as slices. |
| `WriteVectoredTo`     | `write_vectored<T: std::io::Write>(&self, buf: &mut T)` | `std::io::Result<usize>` | Attempts a vectored write of the block (analogous to the standard `write_vectored`). |
| `WriteVectoredTo`     | `write_vectored_all<T: std::io::Write>(&self, buf: &mut T)` | `std::io::Result<()>` | Attempts a vectored write of the block (analogous to the standard `write_vectored_all`). |
| `CrcU32`             | `fn crc(&self)` | `[u8; 4]` | Computes the CRC of the block. |
| `StaticSize`         | `fn ssize()` | `u64` | Returns the size of the block in bytes. |

It is evident that all the listed write methods transmit the binary representation of the block to the output.

As mentioned earlier, `brec` also generates a reference representation of a block:

```ignore
#[block]
pub struct BlockWithEnum {
    pub level: Level,
    pub data: [u8; 200],
}

// Generated by `brec`
struct BlockWithEnumReferred<'a> {
    pub level: Level,
    pub data: &'a [u8; 200],
}
```

As seen above, the reference representation of a block does not store the slice itself but rather a reference to it. This allows the user to inspect the block while avoiding unnecessary data copying. If needed, the referenced block can be easily converted into the actual block using `.into()`, at which point the data will be copied from the source.

### Block Parameters

The `block` macro can be used with the following directives:

- `path = mod::mod` - Specifies the module path for the block if it is not directly imported at the location of `brec::generate!()`. This approach is not recommended (it is better to ensure block visibility at the generator call site), but it is not inherently inefficient or unstable. However, using this method may make future code maintenance more difficult.
- `no_crc` - Disables CRC verification for the block. Note that this does not remove the CRC field from the binary representation of the block. The CRC field will still be present but filled with zeros, and no CRC calculation will be performed.

## Payloads

`brec` does not impose any restrictions on the type of data that can be defined as a payload. However, a payload must implement the following traits:

### Required Traits

| Trait                  | Method | Return Type | Description |
|------------------------|--------|-------------|-------------|
| `PayloadSize`         | `size(&self)` | `std::io::Result<u64>` | Returns the size of the payload body in bytes (excluding the header). |
| `PayloadSignature`    | `sig(&self)` | `ByteBlock` | Returns the signature, which can have a variable length of 4, 8, 16, 32, 64, or 128 bytes. |
| `StaticPayloadSignature` | `ssig()` | `ByteBlock` | Similar to `sig`, but can be called without creating a payload instance. |
| `PayloadEncode`       | `encode(&self)` | `std::io::Result<Vec<u8>>` | Creates a binary representation of the payload (excluding the header). |
| `PayloadEncodeReferred` | `encode(&self)` | `std::io::Result<Option<&[u8]>>` | Creates a reference representation of the payload, if possible. Used for calculation of payload CRC and can boost performance |
| `PayloadDecode<T>`    | `decode(buf: &[u8])` | `std::io::Result<T>` | Attempts to reconstruct the payload from a byte slice. |

### Automatically Implemented Traits

The following traits are automatically applied and do not require manual implementation, though they can be overridden if needed:

| Trait                  | Method | Return Type | Description |
|------------------------|--------|-------------|-------------|
| `PayloadCrc`          | `crc(&self)` | `std::io::Result<ByteBlock>` | Returns the CRC of the payload, which can have a variable length of 4, 8, 16, 32, 64, or 128 bytes. |
| `ReadPayloadFrom`     | `read<B: std::io::Read>(buf: &mut B, header: &PayloadHeader)` | `Result<T, Error>` | Reads the payload from a source. |
| `TryReadPayloadFrom`  | `try_read<B: std::io::Read + std::io::Seek>(buf: &mut B, header: &PayloadHeader)` | `Result<ReadStatus<T>, Error>` | Attempts to read the payload from a source. If there is insufficient data, it returns a corresponding read status instead of an error. |
| `TryReadPayloadFromBuffered` | `try_read<B: std::io::BufRead>(buf: &mut B, header: &PayloadHeader)` | `Result<ReadStatus<T>, Error>` | Similar to `TryReadPayloadFrom`, but returns a reference representation of the payload (if supported) instead of the actual payload. |
| `WritePayloadWithHeaderTo` | `write<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<usize>` | Equivalent to the standard `write` method, returning the number of bytes written. Does not guarantee that data is flushed to the output, so calling `flush` is required if such guarantees are needed. |
| `WritePayloadWithHeaderTo` | `write_all<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<()>` | Equivalent to the standard `write_all` method. |
| `WriteVectoredPayloadWithHeaderTo` | `write_vectored<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<usize>` | Attempts a vectored write of the payload (analogous to the standard `write_vectored`). |
| `WriteVectoredPayloadWithHeaderTo` | `slices(&mut self)` | `std::io::Result<IoSlices>` | Returns the binary representation of the payload as slices. |
| `WriteVectoredPayloadWithHeaderTo` | `write_vectored_all<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<()>` | Attempts a vectored write of the payload (analogous to the standard `write_vectored_all`). |

### Payload Header (`PayloadHeader`)

The payload header is not a generated structure but a static one included in the `brec` crate. `PayloadHeader` is used when writing a payload into a packet (`Packet`) and is always written before the payload body itself. When manually writing a payload to a data source, it is strongly recommended to prepend it with `PayloadHeader` to facilitate further reading from the source.

`PayloadHeader` implements the following traits:

| Trait               | Method | Return Type | Description |
|---------------------|--------|-------------|-------------|
| `ReadFrom`        | `read<T: std::io::Read>(buf: &mut T)` | `Result<Self, Error>` | Attempts to read `PayloadHeader` from a source. |
| `TryReadFrom`     | `try_read<T: std::io::Read + std::io::Seek>(buf: &mut T)` | `Result<ReadStatus<Self>, Error>` | Attempts to read `PayloadHeader`, but if data is insufficient, returns a corresponding read status instead of an error. Also, moves the source's position only on successful reading; otherwise, it remains unchanged. |
| `TryReadFromBuffered` | `try_read<T: std::io::BufRead>(reader: &mut T)` | `Result<ReadStatus<Self>, Error>` | Identical to `TryReadFrom`. |

Thus, if you are manually implementing payload reading from a source, you should first read `PayloadHeader`, and then, using the obtained header, proceed to read the payload itself.

`PayloadHeader` does not implement any traits for writing to a source. However, it provides the `as_vec()` method, which returns its binary representation.

### Automatically Supported Payload Types

Out of the box, `brec` supports `String` and `Vec<u8>` as payload types. After code generation, these will be included in the corresponding enumeration:

```ignore
brec::generate!();

pub enum Payload {
    // ..
    // User-defined payloads
    // ..
    // Default payloads
    Bytes(Vec<u8>),
    String(String),
}
```

### Defining Payloads with `bincode`

Enabling the `bincode` feature provides the simplest and most flexible way to define payload types. By specifying `#[payload(bincode)]`, any type that supports `serde` serialization and deserialization can be used as a `payload`.

If you need runtime payload state, see the `Payload Context` section.

If you need transparent payload encryption, use `#[payload(bincode, crypt)]` together with the `crypt` feature.

```ignore
#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub struct MyPayloadStruct {
    // User-defined fields
}

#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub enum MyPayloadEnum {
    // User-defined variants
}
```

### Partial Restrictions on Payload Types

It is important to note that the CRC for a payload is generated twice-once when the payload is converted into bytes and again after extraction (to compare with the CRC stored in the payload header). This imposes certain limitations on CRC verification, as `brec` does not restrict the types of data used in a payload. If a payload contains data types that do not guarantee a strict byte sequence, CRC verification will always fail due to variations in byte order. As a result, extracting such a payload from the stream will become impossible.

A simple example of this issue is `HashMap`, which does not guarantee a consistent field order upon reconstruction. For instance:

```ignore
#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub struct MyPayloadB {
    items: HashMap<String, String>,
}
```

Extracting such a payload will be impossible because the CRC will always be different (except when the number of keys in the map is ≤ 1). This issue can be resolved in several ways:

- The simplest approach is to avoid using "unstable" data types and instead choose a type that guarantees a fixed byte sequence.
- Disable CRC verification for this specific payload using the `no_crc` directive: `#[payload(no_crc)]`.
- Disable automatic CRC calculation and implement the `PayloadCrc` trait manually for the specific payload. This can be done using the `no_auto_crc` directive: `#[payload(bincode, no_auto_crc)]`.

### Payload Parameters

The `payload` macro can be used with the following directives:

- `path = mod::mod` - Specifies the module path for the payload if it is not directly imported at the location of `brec::generate!()`. This approach is not recommended (it is better to ensure payload visibility at the generator call site), but it is not inherently inefficient or unstable. However, using this method may make future code maintenance more difficult.
- `no_crc` - Disables CRC verification for the payload. Note that this does not remove the CRC field from the binary representation of the payload (specifically in `PayloadHeader`). The CRC field will still be present but filled with zeros, and no CRC calculation will be performed.
- `no_auto_crc` - Disables CRC verification for `payload(bincode)`, requiring a manual implementation of the `PayloadCrc` trait. This parameter is only relevant when using the `bincode` feature.
- `bincode` - available only when the bincode feature is enabled. It allows using any structure as a payload as long as it meets the requirements of the bincode crate, i.e., it implements serde serialization and deserialization. Please note that bincode has a number of limitations, which you can review in its official documentation.
- `ctx` - marks a type as payload runtime context instead of a regular payload. Such a type is collected by `brec::generate!()` to build the crate-local `PayloadContext<'a>` enum and is passed by mutable reference into payload encode/decode/size operations.
- `crypt` - available only when the `crypt` feature is enabled and intended to be used together with `bincode` as `#[payload(bincode, crypt)]`. It enables transparent payload encryption/decryption driven through `PayloadContext`.

### Payload Context

`brec` separates packet structure from payload runtime state.

Blocks are context-free. Payloads may require extra runtime data during:

- encoding
- decoding
- size calculation
- packet read/write operations

This runtime data is called `Payload Context`.

#### Why It Exists

Some payloads cannot be encoded or decoded from bytes alone.

Typical examples:

- encryption settings
- decryption settings
- external lookup state
- user-defined runtime options

For the built-in crypto integration that uses this mechanism, see [Payload Encryption (`crypt`)](#payload-encryption-crypt).

Instead of passing an arbitrary generic options type through the whole API, `brec` binds a context type to the payload family through `PayloadSchema`.

#### Core Idea

Each payload defines:

```ignore
pub trait PayloadSchema {
    type Context<'a>;
}
```

All payload-specific traits then receive this context explicitly:

```ignore
fn encode(&self, ctx: &mut Self::Context<'_>) -> std::io::Result<Vec<u8>>;
fn decode(buf: &[u8], ctx: &mut Self::Context<'_>) -> std::io::Result<Self>;
fn size(&self, ctx: &mut Self::Context<'_>) -> std::io::Result<u64>;
```

As a result:

- block logic stays clean
- payload logic can use runtime state when needed
- packet and storage APIs stay explicit about where context is consumed

#### Default Context

If a payload does not need any runtime state, use the default context:

```ignore
type Context<'a> = brec::DefaultPayloadContext;
```

`DefaultPayloadContext` is just `()`.

#### Generated Context

When you use `brec::generate!()`, the macro generates a crate-local `PayloadContext<'a>`.

If there are no custom context entries, it becomes:

```ignore
pub type PayloadContext<'a> = ();
```

If custom context types exist, `generate!()` builds an enum:

```ignore
pub enum PayloadContext<'a> {
    None,
    MyOptions(&'a mut MyOptions),
}
```

The working examples are available in the `examples` directory of the repository.

#### Declaring a Custom Context Type

Mark a type with `#[payload(ctx)]`:

```ignore
use brec::payload;

#[payload(ctx)]
pub struct MyOptions {
    pub prefix: String,
}
```

This type is not treated as a regular payload. It is collected only to build `PayloadContext<'a>`.

In other words, `#[payload(ctx)]` means:

- do not generate a normal payload variant for this type
- do generate a matching `PayloadContext` enum variant
- pass this value by mutable reference during payload operations

#### Manual Payload Implementation with Context

If you implement payload traits manually, you can use the generated context and extract your runtime state.

Important: `#[payload]` already generates part of the boilerplate for the type itself.

In the common manual case you only implement the payload-specific logic such as:

- `PayloadEncode`
- `PayloadEncodeReferred`
- `PayloadDecode<T>`
- `PayloadSize`
- optionally `PayloadCrc`

You do not need to manually reimplement the basic glue that `#[payload]` already provides for the example shape below.

Example:

```ignore
use brec::{PayloadCrc, PayloadDecode, PayloadEncode, PayloadEncodeReferred, PayloadSize};

#[payload(ctx)]
pub struct MyOptions {
    pub prefix: String,
}

impl MyOptions {
    fn extract_prefix<'a>(ctx: &'a mut crate::PayloadContext<'_>) -> std::io::Result<&'a str> {
        match ctx {
            crate::PayloadContext::MyOptions(options) => Ok(options.prefix.as_str()),
            crate::PayloadContext::None => Err(std::io::Error::new(
                std::io::ErrorKind::InvalidInput,
                "MyPayload expects PayloadContext::MyOptions",
            )),
        }
    }
}

#[payload]
pub struct MyPayload {
    pub value: String,
}

impl PayloadEncode for MyPayload {
    fn encode(&self, ctx: &mut Self::Context<'_>) -> std::io::Result<Vec<u8>> {
        let prefix = MyOptions::extract_prefix(ctx)?;
        Ok(format!("{}{}", prefix, self.value).into_bytes())
    }
}

impl PayloadEncodeReferred for MyPayload {
    fn encode(&self, _ctx: &mut Self::Context<'_>) -> std::io::Result<Option<&[u8]>> {
        Ok(None)
    }
}

impl PayloadDecode<MyPayload> for MyPayload {
    fn decode(buf: &[u8], ctx: &mut Self::Context<'_>) -> std::io::Result<MyPayload> {
        let prefix = MyOptions::extract_prefix(ctx)?.to_owned();

        let value = String::from_utf8(buf.to_vec())
            .map_err(|err| std::io::Error::new(std::io::ErrorKind::InvalidData, err))?;

        let value = value
            .strip_prefix(&prefix)
            .unwrap_or(&value)
            .to_owned();

        Ok(MyPayload { value })
    }
}

impl PayloadSize for MyPayload {
    fn size(&self, ctx: &mut Self::Context<'_>) -> std::io::Result<u64> {
        Ok(PayloadEncode::encode(self, ctx)?.len() as u64)
    }
}

impl PayloadCrc for MyPayload {}
```

This is the same pattern used in the real example. The full example can be found in the repository under `examples/ctx`.

#### Passing Context into Readers and Writers

Context is supplied at the operation boundary.

Examples:

```ignore
let packet = Packet::new(
    vec![Block::MyBlock(MyBlock { id: 1 })],
    Some(Payload::MyPayload(MyPayload {
        value: "hello".to_owned(),
    })),
);
let mut ctx = PayloadContext::MyOptions(&mut options);
writer.insert(packet, &mut ctx)?;
```

```ignore
let mut ctx = PayloadContext::MyOptions(&mut options);
for packet in reader.iter(&mut ctx) {
    let packet = packet?;
    // ...
}
```

```ignore
let mut ctx = PayloadContext::MyOptions(&mut options);
match packet_reader.read(&mut ctx)? {
    NextPacket::Found(packet) => {
        let _ = packet;
    }
    _ => {}
}
```

The key idea is:

- payload type owns the serialization logic
- context is created by the caller
- reader and writer APIs receive that context explicitly per operation

#### Practical Rule

Use context only for runtime state that genuinely belongs to payload processing.

Good candidates:

- crypto options
- decode-time lookup state
- encode/decode feature flags

Bad candidates:

- block-level concerns
- global application state unrelated to payload bytes
- values that should be stored inside the payload itself

### Payload Encryption (`crypt`)

`crypt` in `brec` adds transparent payload encryption and decryption.

The feature is payload-oriented:

- packet structure stays the same
- blocks are not encrypted
- only the payload body is encrypted
- encryption/decryption is driven through `PayloadContext`

This means the integration point is small: you keep using `Packet`, `PacketBufReader`, `Writer`, `Reader`, and the payload itself, but the payload is declared with crypto support and read/write operations receive crypto options.

Blocks staying visible is an intentional design decision, not a limitation.

That visibility is useful because blocks are often used for:

- packet filtering
- lightweight identification
- fast search and routing
- pre-decoding inspection

This preserves flexibility: sensitive payload data can be encrypted, while block-level metadata may still drive transport and indexing decisions.

Enabling the `crypt` feature also does not mean the whole protocol must become encrypted.

You can freely design a mixed protocol where:

- many packets stay open and readable
- only selected payloads are declared with `crypt`
- encrypted and non-encrypted packets coexist in the same application flow

That is usually the most reasonable approach: encrypt only the parts that actually carry sensitive data, and keep the rest simple and cheap to process.

#### What the Feature Does

At the API level, `crypt` gives you:

- `EncryptOptions`
- `DecryptOptions`
- `CryptPolicy`
- `BricCryptCodec`
- crypto-aware payload generation through `#[payload(..., crypt)]`

Recommended path:

- use `#[payload(bincode, crypt)]` when possible

That is the most ergonomic path because serialization and deserialization are then provided out of the box, and `crypt` only adds transparent encryption/decryption around the encoded payload bytes.

At runtime, the current implementation uses:

- `ChaCha20Poly1305` for payload encryption
- RSA-OAEP-SHA256 for wrapping the session key
- an internal envelope that stores algorithm/version/session metadata

You usually do not work with the envelope directly. In the common case, `brec` handles it for you through payload encode/decode.

#### Enable the Feature

In `Cargo.toml`:

```ignore
[dependencies]
brec = { path = "../brec", features = ["bincode", "crypt"] }
serde = { version = "1.0", features = ["derive"] }
```

If your payload uses `#[payload(bincode, crypt)]`, you need both:

- `bincode` for the payload serialization format
- `crypt` for encryption/decryption

With `bincode`, the payload gets automatic encode/decode support, so you do not have to manually implement the payload traits just to use encryption.

If you need the runtime-state side of this model first, see [Payload Context](#payload-context).

#### Core Idea

When a payload is declared as:

```ignore
#[payload(bincode, crypt)]
pub struct MyPayload {
    pub message: String,
}
```

the generated crate-local `PayloadContext<'a>` gets crypto variants.

In practice, you pass one of these at the operation boundary:

```ignore
let mut encrypt = PayloadContext::Encrypt(&mut encrypt_options);
let mut decrypt = PayloadContext::Decrypt(&mut decrypt_options);
```

So the mental model is simple:

- writer side uses `EncryptOptions`
- reader side uses `DecryptOptions`
- the payload itself stays regular Rust data

This does not create a conflict for mixed protocols.

- encrypted payloads read `PayloadContext::Encrypt(...)` / `PayloadContext::Decrypt(...)`
- non-encrypted payloads may coexist in the same generated payload family
- plain `#[payload(bincode)]` payloads simply do not use the crypto options stored in the context enum

#### Minimal Working Example

This is the exact usage pattern validated in `examples/crypt`.

```ignore
use brec::prelude::*;
use std::io::Cursor;

const PUBLIC_KEY_PEM: &str = r#"-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----"#;

const PRIVATE_KEY_PEM: &str = r#"-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----"#;

const KEY_ID: &[u8] = b"demo-key";

#[block]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MetaBlock {
    pub request_id: u32,
}

#[payload(bincode, crypt)]
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct GreetingPayload {
    pub message: String,
}

brec::generate!();

fn usage() -> Result<GreetingPayload, Box<dyn std::error::Error>> {
    let original = GreetingPayload {
        message: "hello from encrypted payload".to_owned(),
    };

    let mut packet = Packet::new(
        vec![Block::MetaBlock(MetaBlock { request_id: 7 })],
        Some(Payload::GreetingPayload(original.clone())),
    );

    let mut encrypt = EncryptOptions::from_public_key_pem(PUBLIC_KEY_PEM)?
        .with_key_id(KEY_ID.to_vec());
    let mut encrypt_ctx = PayloadContext::Encrypt(&mut encrypt);

    let mut bytes = Vec::new();
    packet.write_all(&mut bytes, &mut encrypt_ctx)?;

    let mut decrypt = DecryptOptions::from_private_key_pem(PRIVATE_KEY_PEM)?
        .with_expected_key_id(KEY_ID.to_vec());
    let mut decrypt_ctx = PayloadContext::Decrypt(&mut decrypt);

    let mut source = Cursor::new(bytes.as_slice());
    let mut reader = PacketBufReader::new(&mut source);

    let packet = match reader.read(&mut decrypt_ctx)? {
        NextPacket::Found(packet) => packet,
        _ => return Err("packet was not restored".into()),
    };

    match packet.payload {
        Some(Payload::GreetingPayload(payload)) => Ok(payload),
        _ => Err("payload was not restored".into()),
    }
}
```

#### How It Plugs into Payloads

`#[payload(bincode, crypt)]` means:

1. payload data is serialized with `bincode`
2. serialized payload bytes are encrypted before being written
3. encrypted bytes are decrypted before payload decoding

Important implications:

- you do not need to manually call `BricCryptCodec` for normal packet flow
- you do need to provide the right crypto context during read/write
- using the wrong context variant or missing crypto options will fail at runtime
- this requirement applies only to payloads declared with `crypt`; plain payloads can live in the same protocol and ignore the crypto context entirely

#### Writer-side Options

Use `EncryptOptions` on the encoding side.

Common constructors:

- `EncryptOptions::new(public_key)`
- `EncryptOptions::from_public_key_pem(...)`
- `EncryptOptions::from_public_key_pem_file(...)`
- `EncryptOptions::from_certificate_pem(...)`
- `EncryptOptions::from_certificate_pem_file(...)`
- `EncryptOptions::from_pem(...)`
- `EncryptOptions::from_pem_file(...)`

Common mutators:

- `with_key_id(...)`
- `clear_key_id()`
- `with_policy(...)`

Notes:

- the public key may come from raw public-key PEM or from an X509 certificate PEM
- `key_id` is optional, but strongly useful when multiple keys may exist at runtime
- `EncryptOptions` internally reuses parsed key material and caches session state

#### Reader-side Options

Use `DecryptOptions` on the decoding side.

Common constructors:

- `DecryptOptions::new(private_key)`
- `DecryptOptions::from_private_key_pem(...)`
- `DecryptOptions::from_private_key_pem_file(...)`
- `DecryptOptions::from_pem(...)`
- `DecryptOptions::from_pem_file(...)`

Common mutators:

- `with_expected_key_id(...)`
- `clear_expected_key_id()`
- `with_policy(...)`

Notes:

- decryption requires the RSA private key
- if `with_expected_key_id(...)` is set, envelopes without matching `key_id` will be rejected
- `DecryptOptions` caches unwrapped session keys for repeated use

#### `key_id` Behavior

`key_id` is optional metadata embedded into the crypto envelope.

Recommended pattern:

- writer sets `EncryptOptions::with_key_id(...)`
- reader sets `DecryptOptions::with_expected_key_id(...)`

This gives you a cheap guard against decrypting with the wrong logical key configuration.

If the reader expects `key_id` and the envelope:

- has no `key_id`, you get `MissingKeyId`
- has a different `key_id`, you get `KeyIdMismatch`

#### `CryptPolicy`

`CryptPolicy` controls runtime caching behavior:

```ignore
pub struct CryptPolicy {
    pub session_reuse_limit: u32,
    pub decrypt_cache_limit: usize,
}
```

Default values:

- `session_reuse_limit = 100`
- `decrypt_cache_limit = 32`

What they mean:

- `session_reuse_limit`: how many payloads may reuse the same encryption session before a new one is created
- `decrypt_cache_limit`: how many decrypted session entries are kept on the reader side

Why these settings exist:

- wrapping and unwrapping a fresh RSA session key for every single message is expensive
- on high-throughput streams with many small messages, doing that work every time would noticeably reduce throughput
- bounded reuse and bounded decrypt-side caching are a practical compromise between performance and crypto session churn

In practice, values in the `50..100` range are often a sensible starting point for busy message streams when you want to keep throughput stable without letting reuse grow unbounded.

Typical use:

```ignore
let policy = CryptPolicy {
    session_reuse_limit: 200,
    decrypt_cache_limit: 64,
};

let encrypt = EncryptOptions::from_public_key_pem(public_pem)?.with_policy(policy);
let decrypt = DecryptOptions::from_private_key_pem(private_pem)?.with_policy(policy);
```

#### When to Use `BricCryptCodec` Directly

Most users should not call `BricCryptCodec` directly during normal packet I/O.

It is useful when you need payload-level crypto outside packet read/write, for example:

- encrypt raw payload bytes manually
- decrypt previously stored encrypted payload bytes
- inspect or parse the internal crypto envelope

Useful methods:

- `BricCryptCodec::encrypt(...)`
- `BricCryptCodec::decrypt(...)`
- `BricCryptCodec::encrypt_payload(...)`
- `BricCryptCodec::decrypt_payload(...)`
- `BricCryptCodec::parse(...)`
- `BricCryptCodec::format(...)`

#### Error Model

Crypto errors are represented by `CryptError` and convert to `std::io::Error`.

Typical categories:

- invalid key material: `InvalidRsaPublicKeyPem`, `InvalidRsaPrivateKeyPem`
- envelope metadata mismatch: `MissingKeyId`, `KeyIdMismatch`
- cryptographic failure: `EncryptPayloadBody`, `DecryptPayloadBody`, `WrapSessionKey`, `UnwrapSessionKey`
- format mismatch: `UnsupportedEnvelopeVersion`, `UnsupportedAlgorithm`, `MalformedEnvelope`

In normal packet flow these surface as I/O errors, so packet readers and writers stay compatible with the rest of the library API.

#### Practical Rules

- Encrypt only payloads that actually require confidentiality.
- Keep blocks non-sensitive, because blocks remain visible for packet scanning/filtering.
- Reuse `EncryptOptions` and `DecryptOptions` instances when processing many packets; the feature is designed for that.
- Prefer setting `key_id` whenever the application may rotate or select keys dynamically.
- Keep the PEM loading boundary outside hot loops when possible.

#### Reference Points in This Repository

- example usage: `examples/crypt`
- stress coverage: `tests/stress_payloads_crypt`

If you need a valid starting point, `examples/crypt/src/main.rs` is the canonical minimal example for this feature in the repository.

## Packets

Users do not need to define possible packet types since any combination of blocks (up to 255) and a single optional payload constitutes a valid packet.

```ignore
brec::generate!();

let my_packet = Packet::new(
    // You are limited to 255 blocks per packet.
    vec![
        Block::MyBlockA(MyBlockA::default()),
        Block::MyBlockC(MyBlockC::default())
    ],
    // Note: payload is optional
    Some(Payload::MyPayloadA(MyPayloadA::default()))
);
```

### Packet Constraints

- A packet can contain **0 to 255 blocks**.
- A packet can include **0 or 1 payload**.

**Warning!** In most cases, having 1-5 blocks per packet is more than sufficient. A significant number of blocks can lead to an increase in compilation time but will not affect the performance of the compiled code. Therefore, if compilation time is a critical factor, it is recommended to avoid a large number of blocks in packets. 

To clarify, **runtime performance is not affected**, but the compilation time increases because the compiler has to generate multiple implementations for generic types used in `PacketDef` (an internal `brec` structure).

### Packet Trait Implementations

A `Packet` can be used as a standalone unit for data exchange. It implements the following traits:

| Trait                 | Method | Return Type | Description |
|-----------------------|--------|-------------|-------------|
| `ReadFrom`           | `read<T: std::io::Read>(buf: &mut T)` | `Result<Self, Error>` | Attempts to read a packet from a source. |
| `TryReadFrom`        | `try_read<T: std::io::Read + std::io::Seek>(buf: &mut T)` | `Result<ReadStatus<Self>, Error>` | Attempts to read a packet, but if data is insufficient, it returns a corresponding read status instead of an error. Also, moves the source’s position only upon successful reading; otherwise, it remains unchanged. |
| `TryReadFromBuffered` | `try_read<T: std::io::BufRead>(reader: &mut T)` | `Result<ReadStatus<Self>, Error>` | Identical to `TryReadFrom`. |
| `WriteMutTo`         | `write<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<usize>` | Equivalent to the standard `write` method, returning the number of bytes written. Does not guarantee that data is flushed to the output, so calling `flush` is required if such guarantees are needed. |
| `WriteMutTo`         | `write_all<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<()>` | Equivalent to the standard `write_all` method. |
| `WriteVectoredMutTo` | `slices(&mut self)` | `std::io::Result<IoSlices>` | Returns the binary representation of the packet as slices. |
| `WriteVectoredMutTo` | `write_vectored<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<usize>` | Attempts a vectored write of the packet (analogous to the standard `write_vectored`). |
| `WriteVectoredMutTo` | `write_vectored_all<T: std::io::Write>(&mut self, buf: &mut T)` | `std::io::Result<()>` | Attempts a vectored write of the packet (analogous to the standard `write_vectored_all`). |

### Packet Filtering

`Packet` provides a highly useful method: 

```ignore
filtered<R: std::io::Read + std::io::Seek>(
    reader: &mut R, 
    rules: &Rules
) -> Result<LookInStatus<Packet>, Error>
```

This method allows you to "peek" into a packet before processing the payload, which can significantly improve performance when filtering specific packets.

## Code Generation

`brec` generates code in two stages:

- When the `#[block]` macro is used, it generates code specific to the corresponding block.  
  The same applies to the `#[payload]` macro, which generates code related to a specific payload type.

- However, the protocol also requires unified types such as `enum Block` (enumerating all user-defined blocks)  
  and `enum Payload` (enumerating all payloads). These general-purpose enums allow the system to handle  
  various blocks and payloads dynamically.

To generate this unified code, the `generate!()` macro must be invoked:

```ignore
pub use blocks::*;
pub use payloads::*;

brec::generate!();
```

This macro must be called exactly **once per crate** and is responsible for:

- Implementing required `brec` traits for all user-defined `Block` types
- Implementing required `brec` traits for all user-defined `Payload` types
- Generating unified enums for blocks: `enum Block { ... }`
- Generating unified enums for payloads: `enum Payload { ... }`
- Exporting several convenience type aliases to simplify usage

### Generated Aliases
The macro defines the following aliases to reduce verbosity when using `brec` types:

| Alias                    | Expanded to                                                                 |
|-------------------------|------------------------------------------------------------------------------|
| `Packet`                    | `PacketDef<Block, Payload, Payload>`                                        |
| `BorrowedPacketBufReader<'a, R>` | `PacketBufReaderDef<'a, R, Block, BlockReferred<'a>, Payload, Payload>` |
| `PacketBufReader<'a, R>`     | same as `BorrowedPacketBufReader<'a, R>`                                   |
| `PeekedBlocks<'a>`           | `PeekedBlocksDef<'a, BlockReferred<'a>>`                                   |
| `PeekedBlock<'a>`            | `PeekedBlockDef<'a, BlockReferred<'a>>`                                    |
| `BorrowedRules<'a>`          | `RulesDef<Block, BlockReferred<'a>, Payload, Payload>`                     |
| `Rules<'a>`                  | same as `BorrowedRules<'a>`                                                |
| `BorrowedRule<'a>`           | `RuleDef<Block, BlockReferred<'a>, Payload, Payload>`                      |
| `Rule<'a>`                   | same as `BorrowedRule<'a>`                                                 |
| `RuleFnDef<D, S>`            | `RuleFnDef<D, S>`                                                          |
| `BorrowedReader<'a, S>`      | `ReaderDef<S, Block, BlockReferred<'a>, Payload, Payload>`                 |
| `Reader<S>`                  | `ReaderDef<S, Block, BlockReferred<'static>, Payload, Payload>`            |
| `Writer<S>`                  | `WriterDef<S, Block, Payload, Payload>`                                    |

These aliases make it easier to work with generated structures and remove the need to repeat generic parameters.

When `brec` is built with the `observer` feature, the macro also generates:

| Alias                    | Expanded to                                                                 |
|-------------------------|------------------------------------------------------------------------------|
| `SubscriptionUpdate`    | `brec::SubscriptionUpdate`                                                   |
| `SubscriptionErrorAction` | `brec::SubscriptionErrorAction`                                           |
| `Subscription`          | local facade over `SubscriptionDef<Block, BlockReferred<'static>, Payload, Payload>` |
| `FileObserverOptions<S>`| local wrapper over `brec::FileObserverOptions<..., SubscriptionWrapper<S>>` |
| `FileObserver`          | local wrapper over `FileObserverDef<Block, BlockReferred<'static>, Payload, Payload>` |
| `FileObserverStream`    | `brec::FileObserverStreamDef<Block, BlockReferred<'static>, Payload, Payload>` |

`Subscription` uses `on_*` callbacks: `on_update`, `on_packet`, `on_error`, `on_stopped`, `on_aborted`.

### Usage Constraints

- The macro **must only be called once** per crate. Calling it more than once will result in compilation errors due to duplicate types and impls.
- The macro **must see all relevant types** (`Block`, `Payload`) in scope. You must ensure they are visible in the location where you call the macro.

### Visibility Requirements

Ensure that all blocks and payloads are imported at the location where the macro is used:
```ignore
pub use blocks::*;
pub use payloads::*;

brec::generate!();
```

### Parameters

The macro can be used with the following parameters:

- `no_default_payload` - Disables the built-in payloads (`String` and `Vec<u8>`).  
  This has no impact on runtime performance but may slightly improve compile times and reduce binary size.

- `payloads_derive = "Trait"` -  
  By default, `brec` automatically collects all `derive` attributes that are common across user-defined payloads
  and applies them to the generated `Payload` enum.  
  This parameter allows you to **manually** specify additional derives for the `Payload` enum-useful if you are
  only using the built-in payloads (`String`, `Vec<u8>`) and do not define custom ones.

For example,

```ignore
pub use blocks::*;

// You don't define any custom payloads and only want to use the built-in ones (`String`, `Vec<u8>`)
brec::generate!(payloads_derive = "Debug, Clone");
```

```ignore
pub use blocks::*;

// You don't define any payloads and explicitly disable the built-in ones
brec::generate!(no_default_payload);
```

If the user **fully disables** payload support (as in the example above),
the macro will **not generate any packet-related types** (see *Generated Aliases*).

# Protocol Tools

## Reading Mixed and Mono Streams

To read from a data source, `brec` includes the `PacketBufReader<R: std::io::Read>` tool (available after code generation by calling `brec::generate!()`). `PacketBufReader` ensures safe reading from both **pure `brec` message streams** and **mixed data streams** (containing both `brec` messages and arbitrary data).

Below is an example of reading all `brec` messages from a stream while counting the number of "junk" bytes (i.e., data that is not a `brec` message):

```ignore
fn reading<R: std::io::Read>(source: &mut R) -> std::io::Result<(Vec<Packet>, usize)> {
    let mut packets: Vec<Packet> = Vec::new();
    let mut reader: PacketBufReader<_> = PacketBufReader::new(source);
    let ignored: Arc<AtomicUsize> = Arc::new(AtomicUsize::new(0));
    let ignored_inner = ignored.clone();
    
    reader
        .add_rule(Rule::Ignored(brec::RuleFnDef::Dynamic(Box::new(
            move |bytes: &[u8]| {
                ignored_inner.fetch_add(bytes.len(), Ordering::SeqCst);
            },
        ))))
        .unwrap();
    
    loop {
        match reader.read() {
            Ok(next) => match next {
                NextPacket::Found(packet) => packets.push(packet),
                NextPacket::NotFound => {
                    // Data will be refilled on the next call
                }
                NextPacket::NotEnoughData(_needed) => {
                    // Data will be refilled on the next call
                }
                NextPacket::NoData => {
                    break;
                }
                NextPacket::Skipped => {
                    //
                }
            },
            Err(err) => {
                return Err(std::io::Error::new(
                    std::io::ErrorKind::InvalidData,
                    err.to_string(),
                ));
            }
        };
    }
    Ok((packets, ignored.load(Ordering::SeqCst)))
}
```

### Key Features of `PacketBufReader`
- If there is **insufficient data** (`NextPacket::NotEnoughData`), `PacketBufReader` will attempt to load more data on each subsequent call to `read()`.  
- If **no `brec` data is found** in the current `read()` iteration (`NextPacket::NotFound`), `PacketBufReader` will also attempt to load more data on each subsequent `read()`.  

Thus, `PacketBufReader` **automatically manages data loading**, removing the need for users to implement their own data-fetching logic.

### `NextPacket` Read Statuses

| Status                    | Description | Can Continue Reading? |
|---------------------------|-------------|------------------------|
| `NextPacket::Found`       | A packet was successfully found and returned. | ✅ Yes |
| `NextPacket::NotFound`    | No packets were found in the current read iteration. | ✅ Yes |
| `NextPacket::NotEnoughData` | A packet was detected, but there is not enough data to read it completely. | ✅ Yes |
| `NextPacket::Skipped`     | A packet was detected but skipped due to filtering rules. | ✅ Yes |
| `NextPacket::NoData`      | No more data can be retrieved from the source. | ❌ No |

After receiving `NextPacket::NoData`, further calls to `read()` are meaningless, as `PacketBufReader` has exhausted all available data from the source.

### Custom Filtering Rules in `PacketBufReader`

Another key feature of `PacketBufReader` is that users can define **custom rules** to be applied during data reading. These rules can be updated dynamically between `read()` calls using `add_rule` and `remove_rule`.

| Rule                   | Available Data                      | Description |
|------------------------|--------------------------------------|-------------|
| `Rule::Ignored`        | `&[u8]`                              | Triggered when data not related to `brec` messages is encountered. Provides a byte slice of the unrelated data. |
| `Rule::Prefilter`      | `PeekedBlocks<'a>`                   | Triggered when a packet is found and its blocks have been partially parsed in zero-copy mode. This is the cheapest place to decide whether the payload should be parsed at all. |
| `Rule::FilterPayload`  | `&[u8]`                              | Allows peeking into the payload bytes before deserialization. This is especially useful if the payload is, for example, a string - enabling scenarios like substring search. |
| `Rule::FilterPacket`   | `&Packet`                            | Triggered after the packet is fully parsed, giving the user a final chance to accept or reject the packet. |

`PeekedBlocks` is the main user-facing facade for cheap prefiltering. It hides the low-level `BlockReferred<'a>` representation while still allowing advanced access through `PeekedBlock::as_referred()` and `PeekedBlocks::as_slice()` when needed.

The rules `Rule::Prefilter` and `Rule::FilterPayload` are particularly effective at improving performance, as they allow you to skip the most expensive part - parsing the payload - if the packet is not needed.

### Recommended Filtering Flow

In practice, filtering is most effective when it is performed in three stages:

1. `Rule::Prefilter`
   Use this stage to inspect blocks in zero-copy mode and reject obviously irrelevant packets before payload parsing begins.
2. `Rule::FilterPayload`
   Use this stage when the packet passed prefiltering but you still want to inspect raw payload bytes before deserialization.
3. `Rule::FilterPacket`
   Use this stage only when you need access to the fully parsed packet.

This staged approach minimizes unnecessary work:

- `Prefilter` avoids decoding the payload at all.
- `FilterPayload` avoids full payload deserialization when raw bytes are enough.
- `FilterPacket` is the final, most expensive decision point.

### Using `PeekedBlocks`

`PeekedBlocks<'a>` is the main facade for cheap block-based filtering. It intentionally exposes operations in terms of user-defined block types instead of forcing manual matching on `BlockReferred<'a>`.

Available methods:

| Method | Description |
|--------|-------------|
| `has::<T>()` | Returns `true` if at least one block of type `T` is present. |
| `get::<T>()` | Returns the first block of type `T`. |
| `find::<T, _>(predicate)` | Returns the first block of type `T` that satisfies the predicate. |
| `iter_as::<T>()` | Iterates only over blocks of type `T`. |
| `iter()` | Iterates over all blocks as `PeekedBlock`. |
| `nth(index)` | Returns the block view at the specified position. |
| `as_slice()` | Returns the underlying slice of referred blocks. This is the advanced escape hatch. |

Available methods on `PeekedBlock<'a>`:

| Method | Description |
|--------|-------------|
| `as_type::<T>()` | Attempts to view the current block as a concrete block type `T`. |
| `as_referred()` | Returns the underlying referred block. This is the advanced escape hatch. |

#### Example: fast prefilter by a block type

```ignore
reader
    .add_rule(Rule::Prefilter(brec::RuleFnDef::Dynamic(Box::new(
        move |blocks| blocks.has::<Metadata>(),
    ))))
    .unwrap();
```

#### Example: find a specific block and inspect its fields

```ignore
reader
    .add_rule(Rule::Prefilter(brec::RuleFnDef::Dynamic(Box::new(
        move |blocks| {
            blocks
                .find::<Metadata, _>(|meta| matches!(meta.level, Level::Err))
                .is_some()
        },
    ))))
    .unwrap();
```

#### Example: iterate only over one block type

```ignore
reader
    .add_rule(Rule::Prefilter(brec::RuleFnDef::Dynamic(Box::new(
        move |blocks| {
            blocks
                .iter_as::<Metadata>()
                .any(|meta| matches!(meta.level, Level::Warn | Level::Err))
        },
    ))))
    .unwrap();
```

### When to use advanced access

Most users should stay within `has`, `get`, `find`, and `iter_as`.

Use `PeekedBlock::as_referred()` or `PeekedBlocks::as_slice()` only when:

- you need direct access to the generated `BlockReferred<'a>` enum,
- you want to perform custom matching not covered by the typed helpers,
- or you are building advanced abstractions on top of `brec`.

That keeps the common filtering path compact while still preserving the full low-level API when needed.

## `brec` Message Storage

In addition to stream reading, `brec` provides a tool for storing packets and accessing them efficiently - `Storage<S: std::io::Read + std::io::Write + std::io::Seek>` (available after invoking `brec::generate!()`).

| Method                                | Description |
|--------------------------------------|-------------|
| `insert(&mut self, packet: Packet)`  | Inserts a packet into the storage. |
| `add_rule(&mut self, rule: Rule)`    | Adds a filtering rule. |
| `remove_rule(&mut self, rule: RuleDefId)` | Removes a filtering rule. |
| `count(&self)` | Returns the number of records currently stored. |
| `iter(&mut self)`                    | Returns an iterator over the storage. This method does not apply filters, even if previously added. |
| `filtered(&mut self)`                | Returns an iterator with filters applied (if any were set via `add_rule`). The filtering rules used in `Storage` are identical to those used in `PacketBufReader`. |
| `nth(&mut self, nth: usize)`         | Attempts to read the packet at the specified index. Note that this method does not apply any filtering, even if filters have been previously defined. |
| `range(&mut self, from: usize, len: usize)` | Returns an iterator over a given range of packets. |
| `range_filtered(&mut self, from: usize, len: usize)` | Returns an iterator over a range of packets with filters applied (if previously set via `add_rule`). |

Filtering by blocks or payload improves performance by allowing the system to avoid fully parsing packets unless necessary.

### Storage Layout and Slot Design

The core design of `Storage` is based on how it organizes packets internally:
- Packets are not stored sequentially but are grouped into **slots**, with **500 packets per slot**.
- Each slot stores metadata about packet positions in the file and includes a **CRC** for slot validation, which makes the storage robust against corruption.
- Thanks to the slot metadata, `Storage` can **quickly locate packets by index** or **return a packet range efficiently**.

As previously mentioned, each slot maintains its own **CRC** to ensure data integrity. However, even if the storage file becomes corrupted and `Storage` can no longer operate reliably, packets remain accessible in a **manual recovery mode**. For example, you can use `PacketBufReader` to scan the file, ignoring slot metadata and extracting intact packets sequentially.

## File Observation

When `brec` is built with the `observer` feature, it can watch a storage file and react to newly appended packets.

There are two public facades generated for this:

- `FileObserver` - callback-based consumption through `Subscription`
- `FileObserverStream` - Tokio stream of observer events

### Callback-based Observation

Use `FileObserver` when you want push-style handling through a subscription object:

```ignore
struct MySubscription;

impl Subscription for MySubscription {
    fn on_update(&mut self, total: usize, added: usize) -> SubscriptionUpdate {
        let _ = (total, added);
        SubscriptionUpdate::Read
    }

    fn on_packet(&mut self, packet: Packet) {
        let _ = packet;
    }
}

let options = FileObserverOptions::new(path).subscribe(MySubscription);
let mut observer = FileObserver::new(options)?;
```

`Subscription` uses `on_*` callbacks:

- `on_update`
- `on_packet`
- `on_error`
- `on_stopped`
- `on_aborted`

### Stream-based Observation

Use `FileObserverStream` when you want pull-style integration in Tokio code:

```ignore
use tokio_stream::StreamExt;

let mut stream = FileObserverStream::new(path)?;

while let Some(event) = stream.next().await {
    match event {
        brec::FileObserverEvent::Packet(packet) => {
            let _ = packet;
        }
        brec::FileObserverEvent::Update { total, added } => {
            let _ = (total, added);
        }
        brec::FileObserverEvent::Error(err) => {
            eprintln!("{err}");
        }
        brec::FileObserverEvent::Stopped(reason) => {
            let _ = reason;
            break;
        }
        brec::FileObserverEvent::Aborted => {
            break;
        }
    }
}
```

### Important Runtime Note

The observer integrates well with Tokio and exposes an async-friendly API, but at the low level it still relies on synchronous, blocking file I/O (`std::fs::File`, `Read`, `Seek`) through the storage reader.

In other words:

- async orchestration: yes
- true non-blocking disk access: no

This is the current design and should be kept in mind when embedding the observer into latency-sensitive async workflows.

# Protocol Specification

## Block

A block supports fields of the following types:

| Type   | Size in Binary Format (bytes) |
|--------|-------------------------------|
| u8     | 1                             |
| u16    | 2                             |
| u32    | 4                             |
| u64    | 8                             |
| u128   | 16                            |
| i8     | 1                             |
| i16    | 2                             |
| i32    | 4                             |
| i64    | 8                             |
| i128   | 16                            |
| f32    | 4                             |
| f64    | 8                             |
| bool   | 1                             |
| [u8; n] | n                            |

Any structure marked with the `block` macro will have the following extended representation in binary format:

| Field                     | Type             |
|---------------------------|------------------|
| Signature                 | [u8; 4]          |
| User-defined fields       | Available types  |
| CRC                       | [u8; 4]          |

Thus, the total binary length of a block is calculated as:

```ignore
length = 4 (Signature) + Block's Fields Length + 4 (CRC)
```

The block signature is generated automatically based on its name (including the module path) and the names of all its fields. The signature hash is computed using a 32-bit algorithm.

The block's CRC (32-bit) is generated based on the values of user-defined fields, excluding the signature and the CRC itself.

## Payload

Any data type that implements the `PayloadEncode` and `PayloadDecode<T>` traits can be used as a payload. These traits handle the conversion of a payload to bytes and its unpacking from bytes, respectively. When serializing a payload into binary format, `brec` automatically adds a `PayloadHeader` to each payload.

### `PayloadHeader` Structure

| Field                 | Size           | Description |
|-----------------------|---------------|-------------|
| Signature Length     | 1 byte        | Length of the signature: 4, 8, 16, 32, 64, or 128 bytes |
| Signature           | 4 to 128 bytes | Unique signature of the payload |
| CRC Length         | 1 byte        | Length of the CRC: 4, 8, 16, 32, 64, or 128 bytes |
| CRC                | 4 to 128 bytes | CRC checksum of the payload |
| Payload Body Length | 4 bytes       | Length of the payload body (`u32`) |

As seen in the `PayloadHeader` structure, it does not have a fixed size since the lengths of the signature and CRC can vary. For example, when using the `bincode` feature, both the signature and CRC lengths are set to 4 bytes (32 bits). However, users can implement their own versions with different lengths.

Thus, any payload in binary format is represented as follows:

| Component       | Size           | Description |
|---------------|---------------|-------------|
| `PayloadHeader` | 14 - 262 bytes | Header containing metadata about the payload |
| Payload's body | ---           | Payload data encoded using `PayloadEncode` |

The CRC of the payload is generated based on the bytes produced by `PayloadEncode`. This introduces some constraints on CRC verification since `brec` does not restrict the types of data used in a payload. If a payload contains data types that do not guarantee a strict byte sequence, CRC verification will always fail due to byte order variations. As a result, extracting the payload from the data stream will become impossible.

A simple example of such a situation is a `HashMap`, which does not guarantee a consistent field order when reconstructed. For instance, defining a payload like this:

```ignore
#[payload(bincode)]
#[derive(serde::Deserialize, serde::Serialize)]
pub struct MyPayloadB {
    items: HashMap<String, String>,
}
```

would make it impossible to extract this payload, as the CRC would always be different (except when the number of keys in the map is ≤ 1). This issue can be resolved in several ways:

- The simplest approach is to avoid using "unstable" data types and instead choose one that guarantees a fixed byte sequence.
- Disable CRC verification for this specific payload by using the `no_crc` directive: `#[payload(bincode, no_crc)]`.
- Disable automatic CRC calculation and implement the `PayloadCrc` trait manually for the specific payload. Automatic CRC calculation can be disabled using the `no_auto_crc` directive: `#[payload(bincode, no_auto_crc)]`.

## Packet

A packet serves as a container for storing a set of blocks and optionally a single payload. Each packet includes its own header, `PacketHeader`:

### `PacketHeader` Structure

| Field                   | Size   | Description |
|-------------------------|--------|-------------|
| Signature              | 8 bytes  | Static packet signature |
| Size                   | 8 bytes  | Total size of the packet (excluding the `PacketHeader`) |
| Block's length         | 8 bytes  | Total length of all blocks (including signatures and CRC) in the packet |
| Payload existence flag | 1 byte   | `0` - packet without payload; `1` - packet contains a payload |
| CRC                    | 4 bytes  | CRC of the `PacketHeader` (not the entire packet, only the header) |

Thus, in binary format, a packet is structured as follows:

| Component       | Size    | Count |
|---------------|--------|------|
| `PacketHeader` | 29 bytes | 1 |
| `Block`        | ---    | 0 to 255 |
| `Payload`      | ---    | 0 or 1 |

# Ensuring `brec` Stability

The stability of `brec` is ensured through two levels of testing.

## Functional Testing

To test reading, writing, parsing, filtering, and other functions, `brec` uses `proptest` to generate **random values** for predefined (structurally consistent) blocks and payloads. Blocks and payloads are tested **both separately and in combination** as part of packet testing. 

Packets are constructed with **randomly generated blocks and payloads**. Additionally, the ability of `brec` tools to **reliably read and write randomly generated blocks** is also tested, specifically focusing on `Storage<S: std::io::Read + std::io::Write + std::io::Seek>` and `PacketBufReader`.

In total, **over 40 GB of test data** is generated for this type of testing.

## Macro Testing

To validate the behavior of the `block` and `payload` macros, `brec` also uses `proptest`, but this time it **not only generates random data but also randomly constructs block and payload structures**.

Each randomly generated set of structures is saved as a separate crate. After generating these test cases, each one is **compiled and executed** to ensure stability. Specifically, all randomly generated packets **must be successfully encoded and subsequently decoded without errors**.

## Performance and Efficiency

To evaluate the performance of the protocol, the following data structure is used:

```ignore
pub enum Level {
    Err,
    Warn,
    Info,
    Debug,
}

pub enum Target {
    Server,
    Client,
    Proxy,
}

#[block]
pub struct Metadata {
    pub level: Level,
    pub target: Target,
    pub tm: u64,
}
```

Note: Conversion of `Level` and `Target` into `u8` is required but omitted here for brevity.

Each packet consists of a `Metadata` block and a `String` payload. Data is randomly generated, and a special "hook" string is inserted randomly into some messages for use in filtering tests.

### Test Description

- **Storage**: Data is written using the `brec` storage API - `Storage<S: std::io::Read + std::io::Write + std::io::Seek>` - and then read back using the same interface.
- **Binary Stream**: Data is written to the file as a plain stream of packets, without slots or metadata. Then it is read using `PacketBufReader`.
- **Streamed Storage**: Data is written using `Storage`, but read using `PacketBufReader`, which ignores slot metadata (treating it as garbage).
- **Plain Text**: Raw text lines are written to the file, separated by `\n`.
- **JSON**: The structure shown above is serialized to JSON using `serde_json` and written as one JSON object per line. During reading, each line is deserialized back to the original structure.

Each test is run in two modes:
- **Reading** - reading all available data.
- **Filtering** - reading only records that match specific criteria: logs of type "error" and containing a search hook in the payload.

**Plain Text** is used as a baseline due to its minimal overhead - raw sequential file reading with no parsing or decoding.  
However, `brec` performance is more meaningfully compared with **JSON**, which also involves deserialization.  
JSON is considered a strong baseline due to its wide use and mature, highly optimized parser.

### Important Notes

- For fairness, **CRC checks are enabled** for all `brec` component. CRC is calculated for blocks, payloads, and slots (in the case of storage).
- Each test is repeated multiple times to produce averaged values (`Iterations` column).

### Test Results

| Test             | Mode      | Size    | Rows       | Time (ms) | Iterations |
|------------------|-----------|---------|------------|-----------|------------|
| Storage          | Filtering | 908 Mb  | 140,000     | 612       | 10         |
| Storage          | Reading   | 908 Mb  | 1,000,000   | 987       | 10         |
| JSON             | Reading   | 919 Mb  | 1,000,000   | 597       | 10         |
| JSON             | Filtering | 919 Mb  | 140,000     | 608       | 10         |
| Binary Stream    | Reading   | 831 Mb  | 1,000,000   | 764       | 10         |
| Binary Stream    | Filtering | 831 Mb  | 140,000     | 340       | 10         |
| Plain Text       | Reading   | 774 Mb  | 1,000,000   | 247       | 10         |
| Plain Text       | Filtering | 774 Mb  | 150,000     | 276       | 10         |
| Streamed Storage | Filtering | 908 Mb  | 140,000     | 355       | 10         |
| Streamed Storage | Reading   | 908 Mb  | 1,000,000   | 790       | 10         |

### Observations

- **Plain text** is the fastest format by nature and serves as a baseline.
- **Storage** gives the slowest reading time in full-scan mode - which is expected due to CRC verification and slot parsing.
- However, when **filtering is enabled**, storage is **only 4ms slower than JSON**, which is a **negligible difference**, especially considering that storage data is CRC-protected and recoverable.
- If the storage file is damaged, packets can still be recovered using `PacketBufReader`, even if the slot metadata becomes unreadable.
- **Binary stream mode** (stream writing and reading with `PacketBufReader`) shows exceptional filtering performance - nearly **twice as fast as JSON** - and even full reading is only slightly slower than JSON (~167ms on 1 GB), which is not significant in most scenarios.

This efficiency is possible because `brec`'s architecture allows it to skip unnecessary work. In contrast to JSON, where every line must be deserialized, `brec` can **evaluate blocks before parsing payloads**, leading to better filtering performance.