Crate codeq

A binary codec library optimized for storage and networking.

This crate provides traits and implementations for encoding and decoding data in a binary format, with special focus on:

• Data integrity through CRC32 checksums
• Efficient handling of file/buffer offsets and sizes

Core Traits

• Codec, Encode, Decode: Main trait for types that can be encoded/decoded
• FixedSize: For types with known encoded size
• Span: For types representing a region in a file/buffer

Key Types

• WithChecksum<T>: Wraps data with CRC32 checksum for integrity
• Offset: Type-safe byte position in a file/buffer
• Size: Type-safe byte length
• Segment<T>: Represents a typed region with offset and size

Utilities

• ChecksumReader/ChecksumWriter: I/O wrappers that calculate checksums
• OffsetReader/OffsetWriter: I/O wrappers that track current position

Examples

Basic encoding and decoding:

use codeq::{Codec, Decode, Encode, WithChecksum};

#[derive(Debug, Clone, PartialEq)]
struct Record {
    id: u32,
    data: Vec<u8>,
}

impl Encode for Record {
    fn encode<W: io::Write>(&self, mut writer: W) -> io::Result<usize> {
        let mut n = 0;
        n += self.id.encode(&mut writer)?;
        n += self.data.encode(&mut writer)?;
        Ok(n)
    }
}
impl Decode for Record {
    fn decode<R: io::Read>(mut reader: R) -> io::Result<Self> {
        Ok(Self {
            id: u32::decode(&mut reader)?,
            data: Vec::decode(&mut reader)?,
        })
    }
}

// Add checksum protection
let record = Record { id: 1, data: vec![1, 2, 3] };
let protected = WithChecksum::new(&record);

let mut buf = Vec::new();
protected.encode(&mut buf).unwrap();
assert_eq!(buf, vec![ //
    0, 0, 0, 1, // id
    0, 0, 0, 3, 1, 2, 3, // data
    0, 0, 0, 0, 31, 101, 71, 147 // checksum
]);

let decoded = Record::decode(&mut buf.as_slice()).unwrap();
assert_eq!(record, decoded);

Differences from serde

While serde is a general-purpose serialization framework, this crate is specialized for:

1. Single Format: Only supports a single, efficient binary format
2. Data Integrity: Built-in checksum support for detecting corruption
3. File Operations: First-class support for file offsets and regions
4. Simple Implementation: Direct encoding/decoding without intermediate formats

Choose this crate when you need:

• Efficient binary serialization with checksums
• Direct control over the binary format
• File/buffer position tracking
• Simple implementation without format abstraction

Use serde when you need:

• Multiple format support (JSON, YAML, etc.)
• Derive macros for automatic implementation
• Complex data structure serialization
• Format-agnostic code

Modules

error_context_ext

Add context to error messages.

testing

Test utilities for codec implementations.

Structs

ChecksumReader

A reader wrapper that calculates CRC32 checksum while reading data.

ChecksumWriter

A writer that calculates CRC32 checksum while writing data.

Offset

Represents a byte position/offset in a file or buffer.

OffsetReader

A reader that tracks the number of bytes read.

OffsetWriter

A writer that tracks the number of bytes written.

Segment

Represents a contiguous region in a file or buffer with an offset and size.

Size

Represents a size in bytes.

WithChecksum

A wrapper that appends a CRC32C checksum to the encoded data.

Traits

Codec

A trait that is Encode and Decode.

Decode

A trait that can be decoded from an io::Read stream.

Encode

A trait that can be encoded into an io::Write stream.

FixedSize

Trait for types that have a fixed, known size when encoded.

OffsetSize

A trait for types that span a range with an offset and size

Span

A trait for types that span a range with an offset and size