Crate lencode 

Compact, fast binary encoding with varints and optional deduplication.

This crate provides two core traits, Encode and Decode, for serializing types to a Write and deserializing from a Read without relying on std. Integer types use a compact variable‑length scheme (see Lencode) that encodes small values in a single byte while remaining efficient for large values.

Optional deduplication can be enabled per encode/decode call via DedupeEncoder/DedupeDecoder, which replaces repeated values with small IDs to reduce size for data with many duplicates.

Derive macros for Encode and Decode are available from the companion crate lencode_macros and re‑exported in prelude.

Bytes and strings are compacted using a flagged header with opportunistic zstd compression:

• Formats: &[u8], Vec<u8>, VecDeque<u8>, &str, String
• Wire: varint((payload_len << 1) | flag) + payload
  • flag = 1 → payload is a zstd frame (original size is stored in the frame)
  • flag = 0 → payload is raw bytes/UTF‑8
• The encoder picks whichever is smaller per value.

This keeps headers minimal while improving size significantly for repetitive content, and is no_std compatible via zstd-safe.

Quick start:

use lencode::prelude::*;

#[derive(Encode, Decode, PartialEq, Debug)]
struct Point { x: u64, y: u64 }

let p = Point { x: 3, y: 5 };
let mut buf = Vec::new();
let _n = encode(&p, &mut buf).unwrap();
let q: Point = decode(&mut Cursor::new(&buf)).unwrap();
assert_eq!(p, q);

Collections and primitives:

use lencode::prelude::*;

let values: Vec<u128> = (0..10).collect();
let mut buf = Vec::new();
encode(&values, &mut buf).unwrap();
let roundtrip: Vec<u128> = decode(&mut Cursor::new(&buf)).unwrap();
assert_eq!(values, roundtrip);

Deduplication (smaller output for repeated values):

use lencode::prelude::*;

// A small type we want to dedupe; implements Pack and the dedupe markers.
// Note that this is a toy example, in practice `MyId` would be more
// efficiently encoded using regular lencode encoding because it wraps a u32.
#[derive(Clone, Copy, PartialEq, Eq, Debug, Hash)]
struct MyId(u32);

impl Pack for MyId {
    fn pack(&self, w: &mut impl Write) -> Result<usize> { self.0.pack(w) }
    fn unpack(r: &mut impl Read) -> Result<Self> { Ok(Self(u32::unpack(r)?)) }
}
impl DedupeEncodeable for MyId {}
impl DedupeDecodeable for MyId {}

// Prepare some data with many repeats
let vals = vec![MyId(42), MyId(7), MyId(42), MyId(7), MyId(42), MyId(7), MyId(42)];

// Encode without deduplication
let mut plain = Vec::new();
encode(&vals, &mut plain).unwrap();

// Encode with deduplication enabled
let mut enc = DedupeEncoder::new();
let mut deduped = Vec::new();
encode_ext(&vals, &mut deduped, Some(&mut enc)).unwrap();
assert!(deduped.len() < plain.len());

// Round-trip decoding with a DedupeDecoder
let mut dec = DedupeDecoder::new();
let rt: Vec<MyId> = decode_ext(&mut Cursor::new(&deduped), Some(&mut dec)).unwrap();
assert_eq!(rt, vals);

Modules

dedupe

io

Lightweight, no-std compatible I/O traits and adapters used by the Encode/Decode APIs.

pack

prelude

Convenience re‑exports for common traits, modules and derive macros.

tuples

u256

A compact U256 newtype with varint and endianness support.

varint

Varint traits and helpers used by the Lencode scheme.

Macros

impl_pack_for_endianness_types

Macro to implement the Pack trait for types that implement endian_cast::Endianness. This avoids orphan rule issues by allowing explicit implementations per type.

impl_to_unsigned_signed

Implements ToUnsigned/ToSigned pairwise conversions for signed/unsigned primitives.

impl_unsigned_integer

Implements the internal integer helper traits for the given unsigned types.

Traits

Decode

Trait for types that can be decoded from a binary stream.

Encode

Trait for types that can be encoded to a binary stream.

Functions

decode

Decodes a value of type T from reader using T’s Decode implementation.

decode_ext

Decodes a value with optional deduplication via DedupeDecoder.

encode

Encodes value into writer using the type’s Encode implementation.

encode_ext

Encodes value with optional deduplication via DedupeEncoder.

Type Aliases

Result

Crate‑wide Result that defaults to Error.