qubit-codec 0.3.2

Reusable byte and text codecs for Rust applications
Documentation

Qubit Codec

CircleCI Coverage Status Crates.io Rust License 中文文档

Reusable byte and text codecs for Rust applications.

Overview

Qubit Codec provides small, explicit codecs for stable byte and text encodings commonly needed across Qubit Rust crates and applications. Its API stays lightweight, typed, and idiomatic, with direct concrete methods for common use cases and traits for generic boundaries.

This crate focuses on textual encodings with clear wire-format semantics:

  • hexadecimal byte strings
  • Base64 byte strings
  • C integer literal fragments
  • C string literal byte fragments
  • percent-encoded UTF-8 text
  • application/x-www-form-urlencoded UTF-8 text fragments

It intentionally does not replace Rust's Display, FromStr, TryFrom, or serde APIs for ordinary object conversion.

Design Goals

  • Explicit Semantics: each codec documents its alphabet, separator, padding, and decoding rules.
  • Small API Surface: expose direct encode and decode methods first, with traits available for generic call sites.
  • No Hidden Panics: malformed input is reported as CodecError instead of panicking.
  • Composable Traits: Encoder, Decoder, and Codec support reusable boundaries without forcing dynamic dispatch.
  • Reusable Implementations: common encodings live in one crate instead of being reimplemented by downstream crates.
  • Minimal Dependencies: rely on well-maintained crates only where they add real value.

Features

🔡 Hexadecimal Bytes

  • Lowercase by Default: HexCodec::new() produces contiguous lowercase hex.
  • Uppercase Mode: HexCodec::upper() or with_uppercase(true) produces uppercase digits.
  • Optional Whole Prefix: add and require a prefix such as 0x before the entire encoded value.
  • Optional Per-Byte Prefix: add and require a byte prefix such as 0x before each encoded byte.
  • Optional Separator: write and accept separators between bytes, such as : or a space.
  • Whitespace Handling: optionally ignore ASCII whitespace while decoding.
  • Prefix Case Handling: optionally ignore ASCII case when matching configured prefixes while decoding.
  • Buffer APIs: encode_into and decode_into append into existing buffers.

🔐 Base64 Bytes

  • Standard Alphabet: padded and no-padding standard Base64.
  • URL-Safe Alphabet: padded and no-padding URL-safe Base64.
  • Typed Errors: malformed input is reported as CodecError::InvalidInput.

🔤 C String Literal Bytes

  • Mixed Text and Escapes: decodes fragments such as PK\003\004 and \xd0\xcf.
  • C Escape Support: handles simple, octal, hexadecimal, and universal byte escapes.
  • Byte-Oriented Output: decodes directly to raw bytes without requiring UTF-8.

🔢 C Integer Literals

  • Radix Detection: decodes decimal, octal, and 0x/0X hexadecimal integer literals.
  • Unsigned Output: returns u64 for non-negative integer literal fragments.
  • Precise Errors: reports invalid digits with their original input index.

🌐 Percent-Encoding

  • UTF-8 Text: encodes and decodes UTF-8 strings.
  • RFC 3986 Unreserved Set: leaves ASCII letters, digits, -, ., _, and ~ unchanged.
  • Uppercase Escapes: writes percent escapes such as %2F and %E4.
  • Malformed Escape Detection: reports truncated or invalid %XX sequences.

📝 Form URL Encoding

  • Form Fragment Codec: handles application/x-www-form-urlencoded text fragments.
  • Space as Plus: encodes spaces as + and decodes + back to spaces.
  • Percent Compatibility: shares the same UTF-8 and %XX validation behavior as PercentCodec.

🎯 Focused Public API

  • Encoder<Input>: encodes borrowed input into an associated output type.
  • Decoder<Input>: decodes borrowed input into an associated output type.
  • Codec<EncodeInput, DecodeInput>: combines encoder and decoder traits.
  • CodecError / CodecResult: common error and result types for bundled codecs.

Installation

Add this to your Cargo.toml:

[dependencies]
qubit-codec = "0.3.2"

Quick Start

Hexadecimal Bytes

use qubit_codec::HexCodec;

fn main() {
    let codec = HexCodec::upper()
        .with_prefix("0x")
        .with_separator(" ");

    let encoded = codec.encode(&[0x1f, 0x8b, 0x00, 0xff]);
    assert_eq!("0x1F 8B 00 FF", encoded);

    let decoded = codec
        .decode("0x1F 8B 00 FF")
        .expect("hex text should decode");
    assert_eq!(vec![0x1f, 0x8b, 0x00, 0xff], decoded);
}

Base64 Bytes

use qubit_codec::Base64Codec;

fn main() {
    let codec = Base64Codec::standard();

    let encoded = codec.encode(b"hello");
    assert_eq!("aGVsbG8=", encoded);

    let decoded = codec
        .decode("aGVsbG8=")
        .expect("Base64 text should decode");
    assert_eq!(b"hello".to_vec(), decoded);
}

URL-Safe Base64 Without Padding

use qubit_codec::Base64Codec;

fn main() {
    let codec = Base64Codec::url_safe_no_pad();

    let encoded = codec.encode(&[251, 255, 239]);
    assert_eq!("-__v", encoded);

    let decoded = codec
        .decode("-__v")
        .expect("URL-safe Base64 text should decode");
    assert_eq!(vec![251, 255, 239], decoded);
}

C String Literal Bytes

use qubit_codec::CStringLiteralCodec;

fn main() {
    let codec = CStringLiteralCodec::new();

    let decoded = codec
        .decode(r"PK\003\004")
        .expect("C string literal should decode");
    assert_eq!(b"PK\x03\x04".to_vec(), decoded);

    let encoded = codec.encode(&[0xd0, 0xcf, 0x11, 0xe0]);
    assert_eq!(r"\xD0\xCF\x11\xE0", encoded);
}

C Integer Literals

use qubit_codec::CIntegerLiteralCodec;

fn main() {
    let codec = CIntegerLiteralCodec::new();

    assert_eq!(123, codec.decode("123").expect("decimal should decode"));
    assert_eq!(83, codec.decode("0123").expect("octal should decode"));
    assert_eq!(
        0xbeef_c0de,
        codec.decode("0xBEEFC0DE").expect("hex should decode")
    );
}

Percent-Encoding UTF-8 Text

use qubit_codec::PercentCodec;

fn main() {
    let codec = PercentCodec::new();

    let encoded = codec.encode("a b/中");
    assert_eq!("a%20b%2F%E4%B8%AD", encoded);

    let decoded = codec
        .decode("a%20b%2F%E4%B8%AD")
        .expect("percent-encoded text should decode");
    assert_eq!("a b/中", decoded);
}

Form URL Encoding

use qubit_codec::FormUrlencodedCodec;

fn main() {
    let codec = FormUrlencodedCodec::new();

    let encoded = codec.encode("name=Qubit Codec");
    assert_eq!("name%3DQubit+Codec", encoded);

    let decoded = codec
        .decode("name%3DQubit+Codec")
        .expect("form-url-encoded text should decode");
    assert_eq!("name=Qubit Codec", decoded);
}

Generic Trait Usage

Use the traits when application code should depend on an encoding capability instead of a concrete codec type.

use qubit_codec::{
    CodecError,
    Encoder,
    HexCodec,
};

fn encode_payload<C>(codec: &C, payload: &[u8]) -> Result<String, CodecError>
where
    C: Encoder<[u8], Output = String, Error = CodecError>,
{
    codec.encode(payload)
}

fn main() {
    let text = encode_payload(&HexCodec::new(), &[0xab, 0xcd])
        .expect("hex encoding should not fail");
    assert_eq!("abcd", text);
}

API Reference

Trait Operations

Trait Method Description
Encoder<Input> encode(&Input) Encode borrowed input into an associated output type
Decoder<Input> decode(&Input) Decode borrowed input into an associated output type
Codec<EncodeInput, DecodeInput> - Marker-style combination of Encoder and Decoder

HexCodec Operations

Method Description
new() Create a lowercase codec without prefix or separators
upper() Create an uppercase codec without prefix or separators
with_uppercase(enabled) Configure digit case
with_prefix(prefix) Add and require a whole-output prefix, such as 0x1F8B
with_byte_prefix(prefix) Add and require a prefix before every byte, such as 0x1F 0x8B
with_separator(separator) Add and accept a separator between bytes
with_ignored_ascii_whitespace(enabled) Ignore ASCII whitespace while decoding
with_ignore_prefix_case(enabled) Ignore ASCII case when matching configured prefixes while decoding
encode(bytes) Encode bytes into hexadecimal text
encode_into(bytes, output) Append encoded text into an existing String
decode(text) Decode hexadecimal text into bytes
decode_into(text, output) Append decoded bytes into an existing Vec<u8>

Base64Codec Operations

Method Alphabet Padding Description
standard() Standard Yes Create standard Base64 codec
standard_no_pad() Standard No Create standard Base64 codec without padding
url_safe() URL-safe Yes Create URL-safe Base64 codec
url_safe_no_pad() URL-safe No Create URL-safe Base64 codec without padding
encode(bytes) Configured Configured Encode bytes into Base64 text
decode(text) Configured Configured Decode Base64 text into bytes

CStringLiteralCodec Operations

Method Description
new() Create a C string literal byte codec
encode(bytes) Encode bytes into a C string literal fragment
decode(text) Decode a C string literal fragment into bytes

CIntegerLiteralCodec Operations

Method Description
new() Create a C integer literal decoder
decode(text) Decode a non-negative C integer literal fragment into u64

Text Codec Operations

Type Method Description
PercentCodec new() Create a percent codec
PercentCodec encode(text) Encode UTF-8 text using percent encoding
PercentCodec decode(text) Decode percent-encoded UTF-8 text
FormUrlencodedCodec new() Create a form-url-encoded codec
FormUrlencodedCodec encode(text) Encode UTF-8 text, using + for spaces
FormUrlencodedCodec decode(text) Decode UTF-8 text, treating + as spaces

Error Handling

Bundled decoders return CodecResult<T>, an alias for Result<T, CodecError>.

Error Meaning
MissingPrefix A configured whole or per-byte hex prefix was required but missing
InvalidDigit Input contained a digit that is invalid for the requested radix
InvalidLength Input length does not satisfy a codec requirement
InvalidEscape Input contained a malformed or unsupported escape sequence
InvalidCharacter Input contained a character that cannot appear in that context
InvalidInput Input was rejected by a codec-specific validator
InvalidUtf8 Decoded bytes were not valid UTF-8

Testing & Code Coverage

This project keeps codec behavior covered by integration tests under tests/.

Running Tests

# Run all tests
cargo test

# Run with coverage report
./coverage.sh

# Generate text format report
./coverage.sh text

# Align code with CI requirements
./align-ci.sh

# Run CI checks (format, clippy, test, coverage, audit)
./ci-check.sh

Dependencies

Runtime dependencies are intentionally small:

  • base64 provides the Base64 engines.
  • thiserror provides the public error type implementation.

License

Copyright (c) 2026. Haixing Hu.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

See LICENSE for the full license text.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Development Guidelines

  • Follow Rust API Guidelines.
  • Keep tests comprehensive and deterministic.
  • Document public APIs and behavior changes.
  • Ensure all checks pass before submitting a PR.

Author

Haixing Hu

Related Projects

More Rust libraries from Qubit are available under the qubit-ltd GitHub organization.

Repository: https://github.com/qubit-ltd/rs-codec