PQC Binary Format

A standardized, self-describing binary format for post-quantum cryptography encrypted data interchange.

🌟 The Problem

Post-quantum cryptography (PQC) implementations suffer from the "Babel Tower problem": different implementations cannot interoperate because there is no standardized format for encrypted data. Each library uses its own proprietary format, making cross-platform and cross-language encryption impossible.

💡 The Solution

PQC Binary Format provides a universal, algorithm-agnostic format that:

• ✅ Works across 51 cryptographic algorithms
• ✅ Self-describing metadata enables seamless decryption
• ✅ Integrity verification with SHA-256 checksums
• ✅ Cross-platform compatible (Rust, Python, JavaScript, Go, etc.)
• ✅ Future-proof design allows algorithm migration
• ✅ Minimal dependencies: serde, serde_json, sha2, base64

🚀 Quick Start

Rust

Add to your Cargo.toml:

[dependencies]
pqc-binary-format = "3"

Basic Usage (Rust)

use pqc_binary_format::{PqcBinaryFormat, Algorithm, PqcMetadata, EncParameters};
use std::collections::HashMap;

// Create metadata with encryption parameters
let metadata = PqcMetadata {
    enc_params: EncParameters {
        iv: vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],  // 12-byte nonce
        tag: vec![0; 16],                                  // 16-byte auth tag
        params: HashMap::new(),
    },
    ..Default::default()
};

// Create encrypted data container
let encrypted_data = vec![1, 2, 3, 4, 5];  // Your encrypted bytes
let format = PqcBinaryFormat::new(Algorithm::Hybrid, metadata, encrypted_data);

// Serialize to bytes (for transmission or storage)
let bytes = format.to_bytes().unwrap();

// Deserialize from bytes (includes automatic checksum verification)
let recovered = PqcBinaryFormat::from_bytes(&bytes).unwrap();

assert_eq!(format, recovered);
println!("Algorithm: {}", recovered.algorithm().name());

Python

Install the Python bindings:

cd bindings/python
pip install maturin
maturin develop --release

from pqc_binary_format import Algorithm, EncParameters, PqcMetadata, PqcBinaryFormat

# Create algorithm and metadata
algorithm = Algorithm("hybrid")
enc_params = EncParameters(
    iv=bytes([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]),
    tag=bytes([0] * 16)
)
metadata = PqcMetadata(enc_params=enc_params, kem_params=None, sig_params=None, compression_params=None)

# Create and serialize format
pqc_format = PqcBinaryFormat(algorithm, metadata, bytes([1, 2, 3, 4, 5]))
serialized = pqc_format.to_bytes()

# Deserialize and verify
deserialized = PqcBinaryFormat.from_bytes(serialized)
deserialized.validate()  # Verify checksum integrity
print(f"Algorithm: {deserialized.algorithm.name}")

JavaScript/TypeScript

Build the WebAssembly bindings:

cd bindings/javascript
npm install
npm run build

import init, { WasmAlgorithm, WasmEncParameters, WasmPqcMetadata, WasmPqcBinaryFormat } from './pqc_binary_format.js';

await init();

const algorithm = new WasmAlgorithm('hybrid');
const encParams = new WasmEncParameters(
    new Uint8Array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]),
    new Uint8Array(16)
);
const metadata = new WasmPqcMetadata(encParams);
const pqcFormat = new WasmPqcBinaryFormat(algorithm, metadata, new Uint8Array([1, 2, 3, 4, 5]));

const serialized = pqcFormat.toBytes();
const deserialized = WasmPqcBinaryFormat.fromBytes(serialized);
console.log(`Algorithm: ${deserialized.algorithm.name}`);

Go

Build the Rust library first, then use the Go bindings:

cargo build --release
cd bindings/go
go build example.go

package main

import (
    "fmt"
    "log"
    pqc "github.com/PQCrypta/pqc-binary-format/bindings/go/v3"
)

func main() {
    // Metadata is an opaque, length-prefixed section (UTF-8 JSON by convention).
    metadata := []byte(`{"encryption_params":{"iv":"...","tag":"...","params":{}}}`)
    data := []byte{1, 2, 3, 4, 5}

    format := pqc.New(pqc.AlgorithmHybrid, metadata, data)

    serialized, err := format.Serialize()
    if err != nil {
        log.Fatal(err)
    }
    deserialized, err := pqc.Parse(serialized) // verifies the checksum
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Algorithm: %s (0x%04X)\n", deserialized.AlgorithmName(), deserialized.AlgorithmID)
}

C/C++

Build the Rust library and generate the C header:

cargo build --release
cbindgen --config cbindgen.toml --output include/pqc_binary_format.h
cd bindings/c-cpp
make

#include "pqc_binary_format.h"
#include <iostream>
#include <vector>

int main() {
    std::vector<uint8_t> iv = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};
    std::vector<uint8_t> tag(16, 0);
    std::vector<uint8_t> data = {1, 2, 3, 4, 5};

    PqcFormatHandle* format = pqc_format_new(
        PQC_ALGORITHM_HYBRID,
        iv.data(), iv.size(),
        tag.data(), tag.size(),
        data.data(), data.size()
    );

    ByteBuffer serialized = pqc_format_to_bytes(format);
    PqcFormatHandle* deserialized = pqc_format_from_bytes(serialized.data, serialized.len);

    char* alg_name = pqc_format_get_algorithm_name(deserialized);
    std::cout << "Algorithm: " << alg_name << std::endl;

    pqc_free_string(alg_name);
    pqc_free_buffer(serialized);
    pqc_format_free(deserialized);
    pqc_format_free(format);

    return 0;
}

🌐 Language Bindings

PQC Binary Format provides production-ready, fully tested bindings for multiple programming languages. All bindings support the complete API and produce cross-compatible binary formats.

Available Bindings

Language	Status	Package	Documentation	Examples
Rust	✅ Native	pqc-binary-format	docs.rs	3 examples
Python	✅ Tested	pqc_binary_format	Python README	2 examples
JavaScript/WASM	✅ Tested	pqc-binary-format (npm)	JS README	1 example
Go	✅ Tested	github.com/PQCrypta/pqc-binary-format/bindings/go/v3	pkg.go.dev	1 example
C	✅ Tested	FFI via Rust	C/C++ README	1 example
C++	✅ Tested	FFI via Rust	C/C++ README	1 example

Installation Quick Reference

# Rust
cargo add pqc-binary-format

# Python
pip install pqc-binary-format

# JavaScript/WASM
npm install pqc-binary-format

# Go
go get github.com/PQCrypta/pqc-binary-format/bindings/go/v3

# C/C++ (build from source)
cargo build --release --no-default-features
# Link against target/release/libpqc_binary_format.so

Cross-Language Compatibility

All language bindings are fully interoperable! You can:

• ✅ Encrypt data in Python, decrypt in Rust
• ✅ Serialize in Go, deserialize in JavaScript
• ✅ Create format in C++, validate in Python
• ✅ Mix any combination across platforms

Example workflow:

# Create encrypted data with Python
python3 examples/python/basic_usage.py > data.bin

# Verify with C++
LD_LIBRARY_PATH=target/release ./examples/cpp/basic_usage < data.bin

# Process with Go
cd examples/go && go run basic_usage.go < ../../data.bin

Binding Features

All bindings support:

• ✅ Full algorithm suite (51 algorithms)
• ✅ Metadata serialization/deserialization
• ✅ SHA-256 integrity verification
• ✅ Feature flags (compression, streaming, etc.)
• ✅ Error handling with detailed messages
• ✅ Memory safety (Rust-backed)

Package Distribution Status

Platform	Status	Notes
crates.io (Rust)	✅ Published
PyPI (Python)	✅ Published
npm (JavaScript)	✅ Published
Go module proxy	✅ Published from 3.0.0

📦 Binary Format Specification

All multi-byte integers are little-endian; the metadata section is a UTF-8 JSON object. Fields appear in this exact order (see draft-riddel-pqc-binary-format-00, Section 3):

+-------------------+
| Magic (4 bytes)   | "PQC\x01" - Format identifier
+-------------------+
| Version (1 byte)  | 0x01 - Format version
+-------------------+
| Algorithm (2 bytes)| Algorithm identifier (0x0050 - 0x0905), little-endian
+-------------------+
| Flags (1 byte)    | Feature flags (reserved, 0x00 in v1)
+-------------------+
| Metadata Len (4)  | Length of metadata section, little-endian
+-------------------+
| Metadata (var)    | Algorithm-specific parameters (UTF-8 JSON)
+-------------------+
| Data Len (8)      | Length of encrypted payload, little-endian
+-------------------+
| Data (var)        | Encrypted data
+-------------------+
| Checksum (32)     | SHA-256 over all preceding bytes
+-------------------+

Fixed-header size (excluding the variable metadata/data sections) is 52 bytes. The checksum is SHA-256(magic ‖ version ‖ algorithm_id ‖ flags ‖ metadata_len ‖ metadata ‖ data_len ‖ data); because the JSON metadata is emitted with sorted keys it is deterministic across implementations.

🔐 Supported Algorithms

The format supports 51 cryptographic algorithm identifiers, the registry in src/algorithm.rs (Algorithm::ALL); tests/registry.rs holds this list to it:

Classical Algorithms (0x0050-0x00FF)

• Classical (0x0050): Classical cryptography: X25519 + Ed25519 + AES-256-GCM
• Password Classical (0x0051): Password-based classical encryption

Hybrid Algorithms (0x0100-0x01FF)

• Hybrid (0x0100): Hybrid: ML-KEM-1024 + X25519 + ML-DSA-87 + Ed25519 + AES-256-GCM

Post-Quantum Algorithms (0x0200-0x02FF)

• Post-Quantum (0x0200): Post-quantum: ML-KEM-1024 + ML-DSA-87 + AES-256-GCM
• Multi-Algorithm (0x0201): Multi-algorithm runtime selection
• ML-KEM-1024 (0x0202): ML-KEM-1024 with AES-256-GCM
• Multi-KEM Dual Layer (0x0203): Multi-KEM dual layer
• Multi-KEM Triple Layer (0x0204): Multi-KEM triple layer
• Quad-Layer (0x0205): Quad-layer redundant security
• Lattice-Code Hybrid (0x0206): Lattice-Code hybrid stack
• PQ3-Stack (0x0207): PQ3-Stack with forward secrecy
• X25519MLKEM768 (0x0208): X-Wing hybrid KEM: ML-KEM-768 (FIPS 203) combined with X25519, the pair TLS negotiates as X25519MLKEM768
• FrodoKEM-1344 (0x0209): FrodoKEM-1344: unstructured-lattice KEM, NIST level 5 (ISO/IEC 18033-2)
• Classic-McEliece-6960119 (0x020A): Classic McEliece 6960119: code-based KEM (ISO/IEC 18033-2)
• FrodoKEM-976 (0x020B): FrodoKEM-976: unstructured-lattice KEM, NIST level 3

Max Secure Series (0x0300-0x03FF)

• Max Secure: PQ Lightweight (0x0300): Max Secure: PQ Lightweight
• Max Secure: Pure PQ (0x0301): Max Secure: Pure PQ
• Max Secure: Hybrid (0x0302): Max Secure: Hybrid Transition
• Max Secure: Stateless (0x0303): Max Secure: Stateless
• Max Secure: Crypto-Agile (0x0304): Max Secure: Crypto-Agile
• Max Secure: PQC + ZK (0x0305): Max Secure: PQC + Zero-Knowledge
• Max Secure: Hybrid Transition (0x0306): Max Secure: Hybrid Transition

FN-DSA Series, Falcon-based (0x0400-0x04FF)

• FN-DSA 512: Compact (0x0400): FN-DSA 512: Compact
• FN-DSA 1024: High-Security (0x0401): FN-DSA 1024: High-Security
• FN-DSA: Floating-Point Hardened (0x0402): FN-DSA: Floating-Point Hardened
• FN-DSA: Dual Signature (0x0403): FN-DSA: Dual Signature
• FN-DSA: Transition Stack (0x0404): FN-DSA: Transition Stack
• FN-DSA + ZK Stack (0x0405): FN-DSA + Zero-Knowledge Stack
• FN-DSA + ZK Stack Enhanced (0x0406): FN-DSA + ZK Stack Enhanced
• FN-DSA: Transition Stack Enhanced (0x0407): FN-DSA: Transition Stack Enhanced

Experimental (0x0500-0x05FF)

• Quantum-Inspired Lattice Fusion (0x0500): Quantum-Inspired Lattice Fusion
• Post-ZK Homomorphic (0x0501): Post-ZK Homomorphic with LFHE 2023
• Quantum-Resistant Consensus (0x0502): Quantum-Resistant Consensus
• Entropy-Orchestrated (0x0503): Entropy-Orchestrated PQ Stack
• Lattice-Code Hybrid FN (0x0504): Lattice-Code Hybrid FN
• AI-Synthesized Crypto-Agile (0x0505): AI-Synthesized Crypto-Agile
• Experimental Engine (0x0506): Experimental Engine (generic)

HQC Code-Based Series, NIST 2025 Backup KEM (0x0600-0x06FF)

• HQC-128 (0x0600): HQC-128 (NIST Level 1, 128-bit security)
• HQC-192 (0x0601): HQC-192 (NIST Level 3, 192-bit security)
• HQC-256 (0x0602): HQC-256 (NIST Level 5, 256-bit security)

NIST ML-KEM Variants - FIPS 203 (0x0700-0x07FF)

• ML-KEM-512 (0x0700): ML-KEM-512 (NIST Level 1, 128-bit security)
• ML-KEM-768 (0x0701): ML-KEM-768 (NIST Level 3, 192-bit security)

NIST ML-DSA Variants - FIPS 204 (0x0800-0x08FF)

• ML-DSA-44 (0x0800): ML-DSA-44 (NIST Level 2, 128-bit security)
• ML-DSA-65 (0x0801): ML-DSA-65 (NIST Level 3, 192-bit security)
• ML-DSA-87 (0x0802): ML-DSA-87 (NIST Level 5, 256-bit security)

NIST SLH-DSA Variants - FIPS 205 (0x0900-0x09FF)

• SLH-DSA-SHA2-128s (0x0900): SLH-DSA-SHA2-128s (NIST Level 1, small signatures)
• SLH-DSA-SHA2-128f (0x0901): SLH-DSA-SHA2-128f (NIST Level 1, fast signatures)
• SLH-DSA-SHA2-192s (0x0902): SLH-DSA-SHA2-192s (NIST Level 3, small signatures)
• SLH-DSA-SHA2-192f (0x0903): SLH-DSA-SHA2-192f (NIST Level 3, fast signatures)
• SLH-DSA-SHA2-256s (0x0904): SLH-DSA-SHA2-256s (NIST Level 5, small signatures)
• SLH-DSA-SHA2-256f (0x0905): SLH-DSA-SHA2-256f (NIST Level 5, fast signatures)

View full algorithm list

🎯 Features

Feature Flags

Control optional behavior with feature flags:

use pqc_binary_format::{PqcBinaryFormat, Algorithm, FormatFlags, PqcMetadata, EncParameters};
use std::collections::HashMap;

let flags = FormatFlags::new()
    .with_compression()       // Data was compressed before encryption
    .with_streaming()         // Streaming encryption mode
    .with_additional_auth();  // Additional authentication layer

let metadata = PqcMetadata {
    enc_params: EncParameters {
        iv: vec![1; 12],
        tag: vec![1; 16],
        params: HashMap::new(),
    },
    ..Default::default()
};

let format = PqcBinaryFormat::with_flags(
    Algorithm::QuadLayer,
    flags,
    metadata,
    vec![1, 2, 3],
);

assert!(format.flags().has_compression());
assert!(format.flags().has_streaming());

Metadata Structure

The format includes rich metadata for decryption:

use pqc_binary_format::{PqcMetadata, KemParameters, SigParameters, EncParameters, CompressionParameters};
use std::collections::HashMap;

let metadata = PqcMetadata {
    // Key Encapsulation (optional)
    kem_params: Some(KemParameters {
        public_key: vec![/* ML-KEM public key */],
        ciphertext: vec![/* encapsulated key */],
        params: HashMap::new(),
    }),

    // Digital Signature (optional)
    sig_params: Some(SigParameters {
        public_key: vec![/* ML-DSA public key */],
        signature: vec![/* signature bytes */],
        params: HashMap::new(),
    }),

    // Symmetric Encryption (required)
    enc_params: EncParameters {
        iv: vec![1; 12],              // Nonce/IV
        tag: vec![1; 16],             // AEAD auth tag
        params: HashMap::new(),
    },

    // Compression (optional)
    compression_params: Some(CompressionParameters {
        algorithm: "zstd".to_string(),
        level: 3,
        original_size: 1024,
        params: HashMap::new(),
    }),

    // Custom parameters (extensible)
    custom: HashMap::new(),
};

Custom Parameters

Add your own metadata:

use pqc_binary_format::PqcMetadata;

let mut metadata = PqcMetadata::new();
metadata.add_custom("my_param".to_string(), vec![1, 2, 3]);

// Later...
if let Some(value) = metadata.get_custom("my_param") {
    println!("Custom param: {:?}", value);
}

🔍 Integrity Verification

Every format includes a SHA-256 checksum calculated over all fields:

use pqc_binary_format::PqcBinaryFormat;

let bytes = format.to_bytes().unwrap();

// Tamper with the data
// let mut corrupted = bytes.clone();
// corrupted[50] ^= 0xFF;

// Deserialization automatically verifies checksum
match PqcBinaryFormat::from_bytes(&bytes) {
    Ok(format) => println!("✓ Checksum valid"),
    Err(e) => println!("✗ Checksum failed: {}", e),
}

📚 Examples

Example 1: Basic Encryption Format

use pqc_binary_format::{PqcBinaryFormat, Algorithm, PqcMetadata, EncParameters};
use std::collections::HashMap;

fn main() {
    let metadata = PqcMetadata {
        enc_params: EncParameters {
            iv: vec![1; 12],
            tag: vec![1; 16],
            params: HashMap::new(),
        },
        ..Default::default()
    };

    let format = PqcBinaryFormat::new(
        Algorithm::Hybrid,
        metadata,
        vec![/* your encrypted data */],
    );

    // Save to file
    let bytes = format.to_bytes().unwrap();
    std::fs::write("encrypted.pqc", &bytes).unwrap();

    // Load from file
    let loaded_bytes = std::fs::read("encrypted.pqc").unwrap();
    let loaded = PqcBinaryFormat::from_bytes(&loaded_bytes).unwrap();

    println!("Algorithm: {}", loaded.algorithm().name());
}

Example 2: Cross-Language Interoperability

Rust (Encryption)

let format = PqcBinaryFormat::new(Algorithm::PostQuantum, metadata, data);
let bytes = format.to_bytes().unwrap();
// Send bytes to Python/JavaScript/Go/C++

Python (Decryption)

from pqc_binary_format import PqcBinaryFormat

format = PqcBinaryFormat.from_bytes(bytes)
print(f"Algorithm: {format.algorithm.name}")
print(f"Data: {len(format.data)} bytes")

JavaScript (Decryption)

const format = WasmPqcBinaryFormat.fromBytes(bytes);
console.log(`Algorithm: ${format.algorithm.name}`);
console.log(`Data: ${format.data.length} bytes`);

Go (Decryption)

format, _ := pqc.FromBytes(bytes)
defer format.Free()
fmt.Printf("Algorithm: %s\n", format.GetAlgorithmName())
fmt.Printf("Data: %d bytes\n", len(format.GetData()))

Example 3: Algorithm Migration

// Old data encrypted with Classical algorithm
let old_format = PqcBinaryFormat::from_bytes(&old_encrypted_data)?;
assert_eq!(old_format.algorithm(), Algorithm::Classical);

// Re-encrypt with Post-Quantum algorithm
let plaintext = decrypt_with_classical(&old_format)?;
let new_metadata = create_pq_metadata()?;
let new_format = PqcBinaryFormat::new(
    Algorithm::PostQuantum,
    new_metadata,
    encrypt_with_pq(&plaintext)?,
);

// Same format, different algorithm!

🎓 Use Cases

1. Cross-Platform Encryption

Encrypt in Rust, decrypt in Python, JavaScript, or Go using the same format.

2. Long-Term Archival

Self-describing format ensures data can be decrypted decades later even as algorithms evolve.

3. Algorithm Agility

Switch between algorithms without changing application code.

4. Compliance & Audit

Embedded metadata provides audit trail for regulatory compliance (GDPR, HIPAA, etc.).

5. Research & Benchmarking

Standardized format enables fair comparison of PQC algorithm performance.

🧪 Testing

# Run tests
cargo test

# Run tests with output
cargo test -- --nocapture

# Run specific test
cargo test test_binary_format_roundtrip

📊 Benchmarks

# Run benchmarks
cargo bench

# View benchmark results
open target/criterion/report/index.html

Performance characteristics:

• Serialization: ~50 MB/s for typical payloads
• Deserialization: ~45 MB/s (includes checksum verification)
• Overhead: ~100 bytes + metadata size

🔧 Development

Building from Source

git clone https://github.com/PQCrypta/pqc-binary-format.git
cd pqc-binary-format
cargo build --release

Running Examples

cargo run --example basic_usage
cargo run --example with_compression
cargo run --example cross_platform

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Current Status

• Language Bindings: ✅ Rust (native, tested on Linux, macOS and Windows), ✅ Python, ✅ JavaScript/WASM, ✅ Go, ✅ C/C++: every binding's example is built and run by CI's bindings job on every pull request
• Examples: ✅ 9 validated examples across 6 languages
• Package Distribution: ✅ All platforms published! crates.io, PyPI, npm, pkg.go.dev

Areas for Contribution

• Additional Language Bindings: Java, C#, Ruby, Swift, Kotlin - help us expand!
• Documentation: Tutorials, integration guides, video walkthroughs
• Testing: Additional test cases, fuzzing, property-based testing
• Performance: SIMD optimizations, benchmark improvements
• Standards: Help draft RFC for IETF standardization submission

📄 License

Licensed under either of:

• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)
• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)

at your option.

🙏 Acknowledgments

This format was developed as part of the PQCrypta enterprise post-quantum cryptography platform. Special thanks to:

• NIST Post-Quantum Cryptography Project
• The Rust cryptography community
• Contributors to pqcrypto, ring, and other foundational crates

📖 References

• NIST Post-Quantum Cryptography
• ML-KEM (Kyber) Specification
• ML-DSA (Dilithium) Specification
• PQCrypta Documentation

🔗 Related Projects

• pqcrypto - Rust PQC implementations
• Open Quantum Safe - PQC library collection
• CIRCL - Cloudflare's crypto library

💬 Community & Support

• GitHub Issues: Report bugs
• Discussions: Ask questions
• Website: pqcrypta.com
• Documentation: docs.rs/pqc-binary-format

---

Made with ❤️ by the PQCrypta Community

Securing the future, one byte at a time.