qubit-codec-misc 0.8.3

Miscellaneous byte and text format codecs for Rust applications
Documentation

Qubit Misc Codec

Rust CI Coverage Crates.io Rust License 中文文档

Reusable byte and text codecs for Rust applications.

Overview

Qubit Misc Codec provides small, explicit codecs for stable byte and text encodings commonly needed across Qubit Rust crates and applications. Its API stays lightweight, typed, and idiomatic, with direct concrete methods for common use cases and traits for generic boundaries.

This crate focuses on textual encodings with clear wire-format semantics:

  • hexadecimal byte strings
  • Base64 byte strings
  • C integer literal fragments
  • C string literal byte fragments
  • percent-encoded UTF-8 text
  • application/x-www-form-urlencoded UTF-8 text fragments

It intentionally does not replace Rust's Display, FromStr, TryFrom, or serde APIs for ordinary object conversion.

Design Goals

  • Explicit Semantics: each codec documents its alphabet, separator, padding, and decoding rules.
  • Small API Surface: expose direct encode and decode methods first, with traits available for generic call sites.
  • No Hidden Panics: malformed input is reported as MiscCodecError instead of panicking.
  • Layered Traits: Codec covers low-level single-value or quantum conversion, while ValueEncoder and ValueDecoder remain owned whole-value convenience traits. Generic adapters, buffered engines, and policy hooks live in qubit-codec; import them from the core crate when building custom adapters around these codecs.
  • Reusable Implementations: common encodings live in one crate instead of being reimplemented by downstream crates.
  • Minimal Dependencies: rely on well-maintained crates only where they add real value.

Features

🔡 Hexadecimal Bytes

  • Lowercase by Default: HexCodec::new() produces contiguous lowercase hex.
  • Uppercase Mode: HexCodec::upper() or with_uppercase(true) produces uppercase digits.
  • Optional Whole Prefix: add and require a prefix such as 0x before the entire encoded value.
  • Optional Per-Byte Prefix: add and require a byte prefix such as 0x before each encoded byte.
  • Optional Separator: write and accept separators between bytes, such as : or a space.
  • Whitespace Handling: optionally ignore ASCII whitespace while decoding.
  • Prefix Case Handling: optionally ignore ASCII case when matching configured prefixes while decoding.
  • Named Configuration: reuse a HexCodecConfig through HexCodec::from_config when several codecs share the same policy.
  • Hex buffer APIs: HexCodec::encode_into and HexCodec::decode_into append into existing buffers.

Whitespace policy depends on the separator configuration. With a non-empty separator, each byte requires two adjacent hex digits: 1 f:8b is rejected with separator :. Without a non-empty separator, ignored whitespace may split a byte, so 0x1 f is accepted with byte prefix 0x. Complete structural tokens take precedence over extra ignored whitespace: with ignored ASCII whitespace enabled and separator :, 1f\t : 8b decodes to [0x1f, 0x8b]. In this example, \t denotes an actual tab character. Required separators remain required. Non-whitespace separators are matched exactly; a whitespace-only separator still requires whitespace between bytes, and ignored ASCII whitespace may consume additional adjacent whitespace.

🔐 Base64 Bytes

  • Standard Alphabet: padded and no-padding standard Base64.
  • URL-Safe Alphabet: padded and no-padding URL-safe Base64.
  • Strict Padding: each facade accepts only its configured padding policy; complete groups such as TWFu are valid with either policy. Whitespace and non-canonical trailing bits are rejected.
  • Quantum Core: Base64QuantumCodec handles complete three-byte to four-unit Base64 quanta; final padding stays in the facade/transcoder layer.
  • Typed Errors: malformed input is reported as MiscCodecError::InvalidBase64, retaining the base64::DecodeError source, error category, and input index when available.
  • Three layers: Base64Codec encodes or decodes a whole value; Base64QuantumCodec handles complete three-byte/four-unit quanta; Base64Encoder and Base64Decoder stream complete values across calls.
  • Independent streaming state: construct a direction-specific encoder or decoder directly, or use Base64Codec::encoder() / decoder() to copy its alphabet and padding configuration. Each instance has independent state and must be reset before its first streaming call.
  • Streaming contract: the encoder retains up to two input bytes internally and emits a partial final group from finish. The decoder leaves an incomplete non-EOF suffix unconsumed; the caller must retain it and present it again with more input, or pass it to transcode_eof once EOF is known. Complete only describes the visible input for that call, not source EOF.
  • Capacity and recovery: NeedOutput.required is the minimum total capacity needed from the advanced output position (output_index + progress.written()) for the next step, not the whole-call output length or an additional byte count. When retaining the buffer, its length must cover that position plus required; when switching to an empty buffer at index zero, provide at least required slots. Advance input by progress.read() before retrying. Capacity bounds are conservative, including for empty input. A malformed-input error can follow partial output; the stream is then poisoned and must be reset before reuse. A general driver should use checked arithmetic for the advanced output position and required buffer length, and handle overflow before allocating.
  • Error coordinates: error.domain_error_ref().and_then(|error| error.input_index()) reports a position in the input slice passed to the current call, not a cumulative stream offset. MiscCodecError::InvalidBase64 and its base64::DecodeError source preserve the engine sub-slice position, which can vary with batching, EOF handling, and output capacity. Errors without an exact engine position use the failing segment's start in the outer error. A driver that needs whole-stream positions must track discarded input itself.

For example, decoding __TWFu@AAA from index 2 reports outer position 6 in both a small-output call and an EOF call. The inner error/source position is 0 for the small-output path and 4 for the EOF path; neither is a whole-stream offset.

The encoder reports capacity from the advanced output position. This example retains the first output and retries only the unconsumed input:

This streaming example requires both direct dependencies shown in Installation.

use qubit_codec::TranscodeStatus;
use qubit_codec::Transcoder;
use qubit_codec_misc::Base64Encoder;

let mut encoder = Base64Encoder::standard();
encoder.reset(&mut [], 0)?;
let mut output = [0; 4];
let first = encoder.transcode(b"ManMan", 0, &mut output, 0)?;
assert_eq!((first.read(), first.written()), (5, 4));
let required = match first.status() {
    TranscodeStatus::NeedOutput { required } => required.get(),
    status => panic!("expected NeedOutput, got {status:?}"),
};
let mut larger = [0; 8];
larger[..4].copy_from_slice(&output);
let next = encoder.transcode(b"ManMan", first.read(), &mut larger, first.written())?;
assert_eq!(required, 4);
assert_eq!(next.read(), 1);
assert_eq!(encoder.finish(&mut larger, 8)?, 0);
assert_eq!(&larger, b"TWFuTWFu");
# Ok::<(), Box<dyn std::error::Error>>(())

When using streaming decode hooks that skip or replace malformed percent, form-URL, or C-string input, the codec reports the invalid span visible in the current chunk. Recovery output for malformed input may therefore depend on chunk boundaries. This caveat does not apply to valid input or strict decoding.

🔤 C String Literal Bytes

  • Mixed Text and Escapes: decodes fragments such as PK\003\004 and \xd0\xcf.
  • C Escape Support: handles simple, octal, hexadecimal, and universal byte escapes.
  • Byte-Oriented Output: decodes directly to raw bytes without requiring UTF-8.

🔢 C Integer Literals

  • Radix Detection: decodes decimal, octal, and 0x/0X hexadecimal integer literals.
  • Unsigned Output: returns u64 for non-negative integer literal fragments.
  • Precise Errors: reports invalid digits with their original input index.
  • Restricted Token Contract: accepts complete non-negative tokens only; signs, suffixes, digit separators, and token-stream parsing are not included.
  • Value-Token Decode: remains a ValueDecoder<str> convenience codec because integer literal encoding strategy and token boundaries are not part of the single-value core abstraction yet.

🌐 Percent-Encoding

  • UTF-8 Text: encodes and decodes UTF-8 strings.
  • RFC 3986 Unreserved Set: leaves ASCII letters, digits, -, ., _, and ~ unchanged.
  • Uppercase Escapes: writes percent escapes such as %2F and %E4.
  • Malformed Escape Detection: reports truncated or invalid %XX sequences.

📝 Form URL Encoding

  • Form Fragment Codec: handles application/x-www-form-urlencoded text fragments.
  • Space as Plus: encodes spaces as + and decodes + back to spaces.
  • WHATWG Character Set: leaves only ASCII letters, digits, *, -, ., and _ unescaped.
  • Percent Compatibility: shares the same UTF-8 and %XX validation behavior as PercentCodec.

🎯 Focused Public API

  • ValueEncoder<Input>: encodes borrowed input into an associated output type.
  • ValueDecoder<Input>: decodes borrowed input into an associated output type.
  • Codec with associated Value and Unit: a low-level trait with unsafe encoding and decoding entry points for one value or codec quantum over caller-provided unit buffers. Callers must uphold the documented index and capacity preconditions; implementations use impl Codec, not unsafe impl Codec.
  • CodecValueEncoder<C> / CodecTranscodeEncoder<C> / CodecTranscodeDecoder<C>: default value and buffered adapters available from qubit-codec.
  • TranscodeEncodeEngine / TranscodeEncodeHooks / TranscodeDecodeEngine / TranscodeDecodeHooks: reusable buffered engines and policy hooks available from qubit-codec for custom adapters.
  • MiscCodecError / MiscCodecResult: common error and result types for bundled codecs.

Installation

Add this to your Cargo.toml:

[dependencies]
qubit-codec-misc = "0.8"

For traits, adapters, or streaming APIs imported from qubit_codec, also add qubit-codec as a direct dependency:

[dependencies]
qubit-codec = "0.17"
qubit-codec-misc = "0.8"

Example: Parsing a Magic Rule

An application that has already extracted attributes from XML can use these codecs to parse a magic rule. This is an application composition example, not a complete MIME loader: the library parses syntax and numeric values, while the application checks the real field width and mask length, associates errors with the source rule, and decides whether to load it. A parse failure rejects the rule; it must not silently keep partially parsed values.

Here, number is a separate illustrative numeric field; it does not imply that the string magic and the numeric value belong to the same MIME matcher rule. u16 is for demonstration only; choose the width according to the actual target field.

use qubit_codec_misc::CIntegerLiteralCodec;
use qubit_codec_misc::CStringLiteralCodec;
use qubit_codec_misc::HexCodec;

fn parse_rule(value: &str, number: &str, mask: &str)
    -> Result<(Vec<u8>, u16, Vec<u8>), String>
{
    let bytes = CStringLiteralCodec::new().decode(value).map_err(|e| e.to_string())?;
    let number = CIntegerLiteralCodec::new().decode(number).map_err(|e| e.to_string())?;
    let number = u16::try_from(number).map_err(|_| "value exceeds 16 bits".to_owned())?;
    let mask = HexCodec::new()
        .with_prefix("0x")
        .with_ignore_prefix_case(true)
        .decode(mask).map_err(|e| e.to_string())?;
    if mask.len() != bytes.len() {
        return Err("mask length must equal magic length".to_owned());
    }
    Ok((bytes, number, mask))
}

fn main() -> Result<(), String> {
    let (bytes, number, mask) = parse_rule(r"PK\003\004", "0x1234", "0XFFFFFFFF")?;
    assert_eq!(b"PK\x03\x04".as_slice(), bytes.as_slice());
    assert_eq!(0x1234, number);
    assert_eq!(vec![0xff; 4], mask);
    assert_eq!(parse_rule("PK", "0x10000", "0xFFFF").unwrap_err(),
        "value exceeds 16 bits");
    assert_eq!(parse_rule("PK", "1", "0xFF").unwrap_err(),
        "mask length must equal magic length");
    Ok(())
}

Quick Start

Hexadecimal Bytes

use qubit_codec_misc::HexCodec;

fn main() {
    let codec = HexCodec::upper()
        .with_prefix("0x")
        .with_separator(" ");

    let encoded = codec.encode(&[0x1f, 0x8b, 0x00, 0xff]);
    assert_eq!("0x1F 8B 00 FF", encoded);

    let decoded = codec
        .decode("0x1F 8B 00 FF")
        .expect("hex text should decode");
    assert_eq!(vec![0x1f, 0x8b, 0x00, 0xff], decoded);
}

Base64 Bytes

use qubit_codec_misc::Base64Codec;

fn main() {
    let codec = Base64Codec::standard();

    let encoded = codec.encode(b"hello");
    assert_eq!("aGVsbG8=", encoded);

    let decoded = codec
        .decode("aGVsbG8=")
        .expect("Base64 text should decode");
    assert_eq!(b"hello".to_vec(), decoded);
}

Whole-value conversion and streaming conversion are separate APIs. The whole-value methods below allocate their returned String or Vec; the transcoder methods write to caller-owned buffers.

use qubit_codec::Transcoder;
use qubit_codec_misc::{Base64Codec, Base64Decoder};

let mut decoder = Base64Decoder::standard();
let mut raw = [0; 3];
let n = decoder.transcode_complete_into(b"TQ==", &mut raw).expect("valid Base64");
assert_eq!(&raw[..n], b"M");

let mut encoder = Base64Codec::standard().encoder();
let mut text = [0; 8];
let n = encoder.transcode_complete_into(b"M", &mut text).expect("sufficient bound");
assert_eq!(&text[..n], b"TQ==");

For streaming, call reset before the first chunk. The decoder does not store an incomplete text suffix, so keep unconsumed input and append new source bytes before retrying. Confirm EOF with the source before calling transcode_eof; after a successful final call, call finish.

use qubit_codec::{TranscodeStatus, Transcoder};
use qubit_codec_misc::Base64Decoder;

let mut decoder = Base64Decoder::standard();
decoder.reset(&mut [], 0).expect("reset");
let mut raw = [0; 3];
let p = decoder.transcode(b"TQ", 0, &mut raw, 0).expect("open stream");
assert_eq!(p.read(), 0);
assert!(matches!(p.status(), TranscodeStatus::NeedInput { .. }));
// Keep the unconsumed TQ and append the final == from the source.
let p = decoder.transcode_eof(b"TQ==", 0, &mut raw, 0).expect("final input");
assert_eq!(&raw[..p.written()], b"M");
decoder.finish(&mut [], 0).expect("close stream");

The encoder retains short input internally, so callers do not need to carry a partial byte tail between calls. It emits the final group only from finish.

use qubit_codec::Transcoder;
use qubit_codec_misc::Base64Encoder;

let mut encoder = Base64Encoder::standard();
let mut text = [0; 8];
encoder.reset(&mut text, 0).expect("reset");
assert_eq!(encoder.transcode(b"M", 0, &mut text, 0).unwrap().written(), 0);
assert_eq!(encoder.transcode(b"a", 0, &mut text, 0).unwrap().written(), 0);
assert_eq!(encoder.transcode(b"n", 0, &mut text, 0).unwrap().written(), 4);
assert_eq!(&text[..4], b"TWFu");
assert_eq!(encoder.finish(&mut text, 4).unwrap(), 0);

URL-Safe Base64 Without Padding

use qubit_codec_misc::Base64Codec;

fn main() {
    let codec = Base64Codec::url_safe_no_pad();

    let encoded = codec.encode(&[251, 255, 239]);
    assert_eq!("-__v", encoded);

    let decoded = codec
        .decode("-__v")
        .expect("URL-safe Base64 text should decode");
    assert_eq!(vec![251, 255, 239], decoded);
}

C String Literal Bytes

use qubit_codec_misc::CStringLiteralCodec;

fn main() {
    let codec = CStringLiteralCodec::new();

    let decoded = codec
        .decode(r"PK\003\004")
        .expect("C string literal should decode");
    assert_eq!(b"PK\x03\x04".to_vec(), decoded);

    let encoded = codec.encode(&[0xd0, 0xcf, 0x11, 0xe0]);
    assert_eq!(r"\xD0\xCF\x11\xE0", encoded);
}

Hexadecimal escapes consume one or two digits. Octal escapes consume up to three digits and retain the low eight bits. Universal \u escapes require exactly four hex digits, and \U escapes require exactly eight; both must fit in one byte and do not convert Unicode to UTF-8. Surrounding quotes are not removed. For example, \x123 yields [0x12, b'3'] and \777 yields [0xff]. Raw source text may contain printable ASCII, tab, LF, VT, and FF; other characters must be escaped. An escaped space (\ ) is accepted. For streaming input, call Codec::decode_eof after confirming EOF to resolve format-specific short escapes.

C Integer Literals

use qubit_codec_misc::CIntegerLiteralCodec;

fn main() {
    let codec = CIntegerLiteralCodec::new();

    assert_eq!(123, codec.decode("123").expect("decimal should decode"));
    assert_eq!(83, codec.decode("0123").expect("octal should decode"));
    assert_eq!(
        0xbeef_c0de,
        codec.decode("0xBEEFC0DE").expect("hex should decode")
    );
}

Percent-Encoding UTF-8 Text

use qubit_codec_misc::PercentCodec;

fn main() {
    let codec = PercentCodec::new();

    let encoded = codec.encode("a b/中");
    assert_eq!("a%20b%2F%E4%B8%AD", encoded);

    let decoded = codec
        .decode("a%20b%2F%E4%B8%AD")
        .expect("percent-encoded text should decode");
    assert_eq!("a b/中", decoded);
}

Form URL Encoding

use qubit_codec_misc::FormUrlencodedCodec;

fn main() {
    let codec = FormUrlencodedCodec::new();

    let encoded = codec.encode("name=Qubit Codec");
    assert_eq!("name%3DQubit+Codec", encoded);

    let decoded = codec
        .decode("name%3DQubit+Codec")
        .expect("form-url-encoded text should decode");
    assert_eq!("name=Qubit Codec", decoded);
}

Generic Trait Usage

Use the traits when application code should depend on an encoding capability instead of a concrete codec type.

These traits are defined by the qubit-codec crate, so generic call sites should declare that crate as a direct dependency.

use qubit_codec::ValueEncoder;
use qubit_codec_misc::{
    MiscCodecError,
    HexCodec,
};

fn encode_payload<C>(
    codec: &mut C,
    payload: &[u8],
) -> Result<String, MiscCodecError>
where
    C: ValueEncoder<[u8], Output = String, Error = MiscCodecError>,
{
    codec.encode(payload)
}

fn main() {
    let mut codec = HexCodec::new();
    let text = encode_payload(&mut codec, &[0xab, 0xcd])
        .expect("hex encoding should not fail");
    assert_eq!("abcd", text);
}

Safe Streaming Hex Byte Adapter

Add both direct dependencies:

[dependencies]
qubit-codec = "0.17"
qubit-codec-misc = "0.8"

HexByteCodec decodes one byte at a time. It does not accept HexCodec whole-value prefixes, per-byte prefixes, or separator configuration; those formatting options do not transfer between the codecs. Keep unconsumed input in pending, remove only the read prefix, and treat only the written output prefix as valid. This fixed two-byte output is sufficient for the example and explicitly cannot produce NeedOutput; a general driver must grow or replace its output buffer on NeedOutput and retry with unconsumed input. Complete means the current call consumed its visible input, not that the source reached EOF. Once the source confirms EOF, call transcode_eof with the pending tail, then finish.

use qubit_codec::CodecTranscodeDecoder;
use qubit_codec::TranscodeDecodeError;
use qubit_codec::TranscodeFailure;
use qubit_codec::TranscodeStatus;
use qubit_codec::Transcoder;
use qubit_codec_misc::HexByteCodec;

fn main() {
    let mut decoder = CodecTranscodeDecoder::new(HexByteCodec::new());
    let chunks: [&[u8]; 3] = [b"1", b"f8", b"b"];
    let mut pending = Vec::new();
    let mut output = [0_u8; 2];
    let mut written = 0;

    decoder.reset(&mut output, 0).expect("reset decoder");
    for (index, chunk) in chunks.iter().enumerate() {
        pending.extend_from_slice(chunk);
        let progress = decoder
            .transcode(&pending, 0, &mut output, written)
            .expect("decode available input");
        let read = progress.read();
        written += progress.written();
        pending.drain(..read);
        assert!(!matches!(progress.status(), TranscodeStatus::NeedOutput { .. }));
        if index < 2 {
            assert!(matches!(progress.status(), TranscodeStatus::NeedInput { .. }));
        } else {
            assert_eq!(TranscodeStatus::Complete, progress.status());
        }
    }

    // Complete says this call consumed its visible input; only the source tells us it has ended.
    assert!(pending.is_empty());
    let eof = decoder
        .transcode_eof(&pending, 0, &mut output, written)
        .expect("finish decoding at confirmed EOF");
    assert_eq!(TranscodeStatus::Complete, eof.status());
    written += eof.written();
    assert_eq!([0x1f, 0x8b], output);
    assert_eq!(2, written);
    assert_eq!(0, decoder.finish(&mut output, written).expect("finish adapter"));

    output = [0xa5; 2];
    decoder.reset(&mut output, 0).expect("reset for short input");
    let error = decoder.transcode_eof(b"1", 0, &mut output, 0).unwrap_err();
    assert!(matches!(
        error,
        TranscodeDecodeError::Failure(TranscodeFailure::IncompleteInput {
            input_index: 0,
            required: 2,
            available: 1
        })
    ));
    assert_eq!([0xa5; 2], output);

    decoder.reset(&mut output, 0).expect("reset for invalid input");
    let error = decoder.transcode(b"1g", 0, &mut output, 0).unwrap_err();
    assert!(matches!(
        error.domain_ref(),
        Some(qubit_codec_misc::MiscCodecError::InvalidDigit { index: 1, .. })
    ));
    assert_eq!([0xa5; 2], output);
}

For the related short-escape boundary, see the CStringLiteralCodec EOF short-escape test (the test checks that a short escape is resolved only after EOF is known).

API Reference

Trait Operations

Trait Method Description
ValueEncoder<Input> encode(&Input) Encode borrowed input into an associated output type
ValueDecoder<Input> decode(&Input) Decode borrowed input into an associated output type
Codec with associated Value and Unit decode, encode Convert one value or codec quantum against caller-provided unit buffers

The low-level codecs convert one byte or one complete quantum. HexByteCodec excludes prefixes and separators, which are handled by HexCodec. PercentCodec and FormUrlencodedCodec validate UTF-8 when their whole-value decode helpers return a String. Base64QuantumCodec handles only complete quanta; Base64Codec, Base64Encoder, and Base64Decoder already handle final partial groups and padding according to their configured policies.

HexCodec Operations

Method Description
new() Create a lowercase codec without prefix or separators
upper() Create an uppercase codec without prefix or separators
with_uppercase(enabled) Configure digit case
with_prefix(prefix) Add and require a whole-output prefix, such as 0x1F8B
with_byte_prefix(prefix) Add and require a prefix before every byte, such as 0x1F 0x8B
with_separator(separator) Add and accept a separator between bytes
with_ignored_ascii_whitespace(enabled) Ignore ASCII whitespace while decoding
with_ignore_prefix_case(enabled) Ignore ASCII case when matching configured prefixes while decoding
encode(bytes) Encode bytes into hexadecimal text
encode_into(bytes, output) Append encoded text into an existing String
decode(text) Decode hexadecimal text into bytes
decode_into(text, output) Append decoded bytes into an existing Vec<u8>

HexCodecConfig Operations

Method Description
new() Create the default lowercase, contiguous-hex configuration
with_uppercase(enabled) Configure digit case
with_prefix(prefix) Configure a whole-output prefix
with_byte_prefix(prefix) Configure a prefix before every byte
with_separator(separator) Configure the separator between bytes
with_ignored_ascii_whitespace(enabled) Configure whitespace handling while decoding
with_ignore_prefix_case(enabled) Configure case-insensitive prefix matching
is_uppercase() / prefix() / byte_prefix() Inspect configured output formatting
separator() / ignores_ascii_whitespace() / ignores_prefix_case() Inspect configured decoding policy
HexCodec::from_config(config) Create a codec from the reusable configuration
HexCodec::config() Borrow the codec's configuration

HexByteCodec Operations

Method or Trait Description
new() Create a lowercase single-byte hex codec
upper() Create an uppercase single-byte hex codec
with_uppercase(enabled) Configure digit case
Codec<Value = u8, Unit = u8> Decode or encode one byte as exactly two ASCII hex units

Base64Codec Operations

Method Alphabet Padding Description
standard() Standard Yes Create standard Base64 codec
standard_no_pad() Standard No Create standard Base64 codec without padding
url_safe() URL-safe Yes Create URL-safe Base64 codec
url_safe_no_pad() URL-safe No Create URL-safe Base64 codec without padding
encode(bytes) Configured Configured Encode bytes into Base64 text
decode(text) Configured Configured Decode Base64 text into bytes

Base64Codec is the owned whole-value facade. It does not provide decode_into; use Base64Decoder and Transcoder when decoding into a caller-owned buffer. Base64Codec::encoder() and decoder() create independent streaming states with the facade's configuration. The direction-specific types also expose standard(), standard_no_pad(), url_safe(), and url_safe_no_pad() constructors; their Default is standard padded Base64.

The padded and unpadded modes reject each other's partial-group form. For example, Base64Codec::standard().decode("Zg") and Base64Codec::standard_no_pad().decode("Zg==") both return an error, while both modes accept "TWFu" because it encodes a complete three-byte group and needs no padding.

Base64QuantumCodec Operations

Method Alphabet Units Description
standard() Standard 4 Create a standard Base64 quantum codec
url_safe() URL-safe 4 Create a URL-safe Base64 quantum codec
Codec<Value = [u8; 3], Unit = u8> Configured 4 Encode or decode one complete Base64 quantum without padding finalization

Invalid quantum units now return MiscCodecError::InvalidBase64 with Base64ErrorKind::InvalidByte, rather than InvalidInput { codec: "base64-quantum", .. }. Read input_index and symbol instead of parsing the diagnostic text. The index is relative to the input slice passed to the call; the invalid quantum still consumes four units. Padding is unsupported by this layer, so = is classified as an invalid byte.

CStringLiteralCodec Operations

Method Description
new() Create a C string literal byte codec
encode(bytes) Encode bytes into a C string literal fragment
decode(text) Decode a C string literal fragment into bytes

CIntegerLiteralCodec Operations

Method Description
new() Create a C integer literal decoder
decode(text) Decode a non-negative C integer literal fragment into u64

CIntegerLiteralCodec intentionally remains a value-token decoder. It does not implement Codec<Value = u64, Unit = u8> yet because that would require committing to token boundary and encode-format policy that belongs above the single-value core.

Text Codec Operations

Type Method Description
PercentCodec new() Create a percent codec
PercentCodec encode(text) Encode UTF-8 text using percent encoding
PercentCodec decode(text) Decode percent-encoded UTF-8 text
FormUrlencodedCodec new() Create a form-url-encoded codec
FormUrlencodedCodec encode(text) Encode UTF-8 text, using + for spaces
FormUrlencodedCodec decode(text) Decode UTF-8 text, treating + as spaces

Error Handling

Bundled decoders return MiscCodecResult<T>, an alias for Result<T, MiscCodecError>.

Error Meaning
MissingPrefix A configured whole or per-byte hex prefix was required but missing
InvalidDigit Input contained a digit that is invalid for the requested radix
InvalidLength Input length does not satisfy a codec requirement
InvalidEscape Input contained a malformed or unsupported escape sequence
InvalidCharacter Input contained a character that cannot appear in that context
InvalidInput Input was rejected by a codec-specific validator
InvalidBase64 Base64 decoding failed; includes a structured category and preserves the underlying base64::DecodeError
InvalidUtf8 Decoded bytes were not valid UTF-8

Complete-value helpers report truncated %XX and C escapes as InvalidEscape. Low-level Codec::decode calls use qubit_codec::DecodeFailure::Incomplete for open-stream tails; callers that have confirmed EOF should call Codec::decode_eof so format-specific short escapes can be resolved.

Performance Considerations

Codec implementations operate on borrowed byte slices or strings and return owned output only when the target format requires it. Configuration is stored in small value types, and generic trait use does not require dynamic dispatch.

Run cargo bench --bench codec_benchmarks -- base64-transcoders to measure Base64 owned, caller-buffer, and streaming paths at 64 B and 64 KiB. Streaming cases include 7-byte and 4096-byte chunks; the mode sample includes the other alphabet and padding configurations. Output buffers are allocated before each timed loop. Owned cases include output allocation, so their timings are not a direct measure of codec work alone. The benchmark reports measured throughput when run; no throughput value is assumed here. The fuzz targets under fuzz/ can be compiled with cargo +nightly fuzz build and run with cargo +nightly fuzz run percent_and_form (or the other target names).

Dependencies

Runtime dependencies are intentionally small:

  • base64 provides the Base64 engines.
  • percent-encoding provides the percent escape byte table.
  • thiserror provides the public error type implementation.

The form_urlencoded crate is used only by differential tests and fuzz targets; it is not part of the runtime API.

Testing

# Run tests with the default feature set
cargo test

# Run tests with all declared features
cargo test --all-features

# Project CI checks
./.infra/bin/ci-check.sh

# Check code coverage
./.infra/bin/coverage.sh

License

Copyright (c) 2025 - 2026. Haixing Hu. All rights reserved.

Licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Contributing

Contributions are welcome. Please follow the Rust API guidelines, keep public API documentation and tests current, and run ./.infra/bin/align-ci.sh to format code and ./.infra/bin/ci-check.sh to satisfy CI requirements before submitting a pull request.

Author

Haixing Hu - Qubit Co. Ltd.

Repository: https://github.com/qubit-ltd/rs-codec-misc