Crate byte_array_ops

Expand description

Welcome to the ByteArrayOps Library

§Introduction

This is a no_std-compatible library designed to provide an ergonomic and easy way to conduct operations on byte arrays. It is intended both for general use where data sensitivity is not important and for cryptographic use cases or other cases where privacy may be of concern.

§byte-array-ops

IMPORTANT: Crate Split in v0.3.0

Starting with v0.3.0, security hardening features will be moved to a separate crate. This crate will remain focused on general-purpose byte array operations. See the Security Hardening section for details.

Active Development Warning This library is under heavy active development. Core functionality (type conversions, bitwise operations) is stable and production ready.

§Overview

A no_std-compatible Rust library for ergonomic byte array operations with optional security hardening.

Design Philosophy:

Compile only what you need - Feature flags for graceful degradation. Basic conversions work without any features, bitwise operations require ops_algebra (enabled by default), and security features are opt-in.
Minimal dependencies - Keep compilation fast and dependency tree small for a pleasant development experience.
Test-driven development - No functionality is added without comprehensive test coverage. Untested or experimental features are gated behind the experimental flag (disabled by default and guarded with compile_error!).

§Features

Feature	Purpose	Default	Status	Use Cases
`ops_algebra`	Bitwise operations (XOR, AND, OR, NOT)	Yes	Implemented	Crypto implementations, data masking, general byte manipulation
`ops_simd`	SIMD-optimized operations	No	Planned	High-performance bulk operations
`experimental`	Unstable/experimental features	No	Ongoing	Development and testing only

Note on Feature Selection:

No features needed: Simple conversions (hex ↔ bytes, UTF-8 ↔ bytes) work with default-features = false
ops_algebra (default): Enabled by default for bitwise operations
Security features: Moved to a separate crate in v0.3.0 - see Security Hardening section

§Installation

[dependencies]
byte-array-ops = "0.1.0"

# Or disable default features for minimal build (conversions only)
byte-array-ops = { version = "0.1.0", default-features = false }

# For no_std environments with alloc and operations
byte-array-ops = { version = "0.1.0", default-features = false, features = ["ops_algebra"] }

§Quick Start

See the API documentation for comprehensive examples including:

Creating ByteArrays from hex, binary, UTF-8, and raw bytes
Bitwise operations (XOR, AND, OR, NOT)
Working with iterators
Convenience macros (try_bytes!, try_hex!, try_bin!)

CAVEAT: The try_bytes! macro silently converts to UTF-8 when no format prefix (0x, 0b, 0o) is provided. Use try_hex! or try_bin! for guaranteed hex/binary parsing without format detection.

§Basic Example

use byte_array_ops::ByteArray;
use byte_array_ops::errors::ByteArrayError;
use byte_array_ops::{try_hex, try_bin};

fn main() -> Result<(),ByteArrayError> {

    // From hex string (using parse)
    let from_hex: ByteArray = "0xdeadbeef".parse()?;
    assert_eq!(from_hex.as_bytes(), [0xde, 0xad, 0xbe, 0xef]);

    // Using macros for convenience
    let with_macro = try_hex!("cafe")?;
    assert_eq!(with_macro.as_bytes(), [0xca, 0xfe]);

    let binary = try_bin!("11110000")?;
    assert_eq!(binary.as_bytes(), [0xf0]);

    // From UTF-8 string (no prefix)
    let from_utf8: ByteArray = "hello".parse()?;
    assert_eq!(from_utf8.as_bytes(), b"hello");

    // Bitwise operations (requires ops_algebra feature)
    let a: ByteArray = "0xff00".parse()?;
    let b: ByteArray = "0x0ff0".parse()?;
    let result = a ^ b; // XOR
    assert_eq!(result.as_bytes(), [0xf0, 0xf0]);
    
    Ok(())
}

§`no_std` Support

This library is no_std compatible and requires only the alloc crate. Perfect for:

Embedded systems with allocators (ESP32, ARM Cortex-M with heap)
Bootloaders and kernel development
WebAssembly environments
Any environment where std is unavailable

§Roadmap

Note on version stability: All versions post-0.2.0 are subject to change depending on whether breaking changes are necessary in previous versions. When no more breaking API changes are planned, the “Active Development Warning” banner above will be removed. This is expected to happen well before v1.0.0 as the API matures for ergonomic use.

§v0.1.0 (Old Milestone)

Core functionality with production-ready type conversions and bitwise operations:

Multiple input formats (hex, binary, UTF-8, raw bytes)
Bitwise operations (XOR, AND, OR, NOT)
Comprehensive iterator support
no_std compatibility with alloc

§v0.2.0 (Current)

API refinement and macro ergonomics:

Cleanup API and experiment with most efficient (and most used) APIs
Lay the groundwork for introducing the SecureReallocationProvider trait, which will encompass more secure implementations of vector methods that may require allocation
Introduce helper macros for ByteArray construction

§v0.3.0 (Planned - Possible Breaking Changes)

Security hardening for cryptographic and sensitive data use cases:

Secure memory wiping (zeroize on drop)
Memory locking (prevent swapping to disk)
Constant-time operations (timing attack prevention)
Hardened constructors and secure reallocation
Bugfixes and refinements

§v0.4.0 (Planned - Possible Breaking Changes)

Performance optimization for high-throughput scenarios:

SIMD-accelerated bitwise operations
Benchmark suite and regression testing
Performance tuning for large arrays

§v1.0.0 (Future)

Stable API with long-term compatibility guarantees:

API freeze after real-world usage validation
Security audit (if feasible - even major libraries like RustCrypto often lack formal audits)
Comprehensive test coverage and fuzzing
Multi-platform testing and verification

§Security Hardening

As of v0.3.0, security hardening features have been moved to a separate crate.

§Why the Split?

Due to inherent design requirements, secure and general-purpose byte array operations cannot coexist in the same crate without maintenance issues:

Technical Constraints:

Additive features requirement: Rust features should be additive to avoid breaking changes upon feature activation/deactivation
Type divergence: Constant-time operations require different types and function signatures that are incompatible with general-purpose use
API restrictions: General-purpose use needs to deref Vec<_> to access standard methods, but secure versions must restrict these due to reallocation and data remnants concerns
Security guarantees: Hardened types cannot provide all standard Vec operations without compromising security

§Migration Path

Current users (v0.2.x):

This crate (byte-array-ops) will continue as a general-purpose byte array library
No action needed for general-purpose use cases

Users needing security hardening:

Security features (zeroize, constant-time ops, memory locking, memory encryption) will be available in a separate crate starting v0.3.0
The secure crate will depend on this crate for core functionality
Documentation for the secure crate will be provided upon release

For historical reference on planned security features, see archive/security_features.md.

§License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

§Quick Start Examples

§Creating ByteArrays - Multiple Ways

use byte_array_ops::ByteArray;
use byte_array_ops::errors::ByteArrayError;

fn main() -> Result<(), ByteArrayError> {
    // From hex string (with 0x prefix)
    let from_hex: ByteArray = "0xdeadbeef".parse()?;
    assert_eq!(from_hex.as_bytes(), [0xde, 0xad, 0xbe, 0xef]);

    // From binary string (with 0b prefix)
    let from_bin: ByteArray = "0b11110000".parse()?;
    assert_eq!(from_bin.as_bytes(), [0xf0]);

    // From UTF-8 string (no prefix)
    let from_utf8: ByteArray = "hello".parse()?;
    assert_eq!(from_utf8.as_bytes(), b"hello");

    // From byte slice using From trait
    let from_slice: ByteArray = [0x01, 0x02, 0x03].as_slice().into();
    assert_eq!(from_slice.len(), 3);

    // From `Vec<u8>` using From trait
    let from_vec: ByteArray = vec![0xaa, 0xbb, 0xcc].into();
    assert_eq!(from_vec.as_bytes(), [0xaa, 0xbb, 0xcc]);

    // From array literal using From trait
    let from_array: ByteArray = [0xff; 4].into();
    assert_eq!(from_array.as_bytes(), [0xff, 0xff, 0xff, 0xff]);

    // Using static constructor with hex
    let with_hex = ByteArray::from_hex("cafe")?;
    assert_eq!(with_hex.as_bytes(), [0xca, 0xfe]);

    // Using static constructor with binary
    let with_bin = ByteArray::from_bin("1010_0101")?;
    assert_eq!(with_bin.as_bytes(), [0xa5]);

    Ok(())
}

§Macro Examples

CAVEAT: The try_bytes! macro silently converts to UTF-8 when no format prefix (0x, 0b, 0o) is provided. Use try_hex! or try_bin! for guaranteed hex/binary parsing without format detection.

use byte_array_ops::{try_bytes, try_hex, try_bin};
use byte_array_ops::errors::ByteArrayError;

fn main() -> Result<(), ByteArrayError> {
    // try_bytes! - Parse with format detection (0x/0b prefix or UTF-8)
    let from_hex = try_bytes!("0xdeadbeef")?;
    assert_eq!(from_hex.as_bytes(), [0xde, 0xad, 0xbe, 0xef]);

    let from_bin = try_bytes!("0b11110000")?;
    assert_eq!(from_bin.as_bytes(), [0xf0]);

    // CAVEAT: No prefix = UTF-8 conversion!
    let from_utf8 = try_bytes!("hello")?;
    assert_eq!(from_utf8.as_bytes(), b"hello");

    // try_hex! - Always parses as hex (no UTF-8 fallback)
    let hex_only = try_hex!("cafe")?;
    assert_eq!(hex_only.as_bytes(), [0xca, 0xfe]);

    // try_bin! - Always parses as binary (no UTF-8 fallback)
    let bin_only = try_bin!("11110000")?;
    assert_eq!(bin_only.as_bytes(), [0xf0]);

    Ok(())
}

§XOR Encryption Example

use byte_array_ops::ByteArray;
use byte_array_ops::errors::ByteArrayError;

fn main() -> Result<(), ByteArrayError> {
    // Encrypt plaintext with a key using XOR
    let plaintext: ByteArray = "secret message".parse()?;
    let key: ByteArray = "0x_de_ad_be_ef_ca_fe_ba_be_01_02_03_04_05_06".parse()?;

    let mut cipher_out = vec![0u8; plaintext.len()];
    let encrypted_len = encrypt(plaintext.as_bytes(), key.as_bytes(), &mut cipher_out);

    assert_eq!(encrypted_len, plaintext.len());

    // Decrypt the ciphertext (XOR is symmetric)
    let mut decrypted_out = vec![0u8; encrypted_len];
    let decrypted_len = decrypt(&cipher_out[..encrypted_len], key.as_bytes(), &mut decrypted_out);

    assert_eq!(&decrypted_out[..decrypted_len], plaintext.as_bytes());
    Ok(())
}

fn encrypt(input: &[u8], key: &[u8], output: &mut [u8]) -> usize {
    // Simple XOR encryption (NOT secure, for demonstration only!)
    let input_ba: ByteArray = input.into();
    let key_ba: ByteArray = key.into();
    let result = input_ba ^ key_ba;

    let bytes_written = result.len().min(output.len());
    output[..bytes_written].copy_from_slice(&result.as_bytes()[..bytes_written]);
    bytes_written
}

fn decrypt(input: &[u8], key: &[u8], output: &mut [u8]) -> usize {
    // XOR is symmetric, so decrypt is the same as encrypt
    encrypt(input, key, output)
}

§Bitwise Operations

use byte_array_ops::ByteArray;

fn main() {
    let a: ByteArray = "0xff00".parse().unwrap();
    let b: ByteArray = "0x0ff0".parse().unwrap();

    // XOR operation
    let xor_result = a.clone() ^ b.clone();
    assert_eq!(xor_result.as_bytes(), [0xf0, 0xf0]);

    // AND operation
    let and_result = a.clone() & b.clone();
    assert_eq!(and_result.as_bytes(), [0x0f, 0x00]);

    // OR operation
    let or_result = a.clone() | b.clone();
    assert_eq!(or_result.as_bytes(), [0xff, 0xf0]);

    // NOT operation
    let not_result = !a;
    assert_eq!(not_result.as_bytes(), [0x00, 0xff]);
}

§Module Overview

This library is organized into the following modules:

core::model - Core ByteArray type and constructors
core::type_conv - Type conversion implementations (From, Into, FromStr)
core::ops - Bitwise operations (XOR, AND, OR, NOT) - requires ops_algebra feature
core::iter - Iterator implementations
core::errors - Error types for parsing and conversions
core::macros - Declarative macros for easy ByteArray construction
core::trust::hardened - Security-hardened implementations (when hardening features enabled)
core::trust::insecure - Convenience implementations (default mode)

§Key Types

§Core Types

ByteArray - Main byte array type with automatic capacity management

§Iterators

ByteArrayIter - Immutable iterator over bytes
ByteArrayIterMut - Mutable iterator over bytes

§Error Types

ByteArrayError - Errors during parsing and conversions

§Trait Implementations

ByteArray implements many standard Rust traits for ergonomic usage:

Conversions:

From<&[u8]>, From<Vec<u8>>, From<&Vec<u8>>, From<[u8; N]> - Create from bytes
FromStr - Parse from strings (hex with 0x, binary with 0b, UTF-8 fallback)
Into<Vec<u8>> - Extract underlying bytes (zero-cost move) in insecure mode
AsRef<[u8]> - Borrow as byte slice

Operations (requires ops_algebra feature):

BitXor, BitXorAssign - XOR operations (^, ^=)
BitAnd, BitAndAssign - AND operations (&, &=)
BitOr, BitOrAssign - OR operations (|, |=)
Not - NOT operation (!)

Iteration:

IntoIterator - Iterate by reference (&ByteArray, &mut ByteArray)
ExactSizeIterator - Iterators with known length

Comparison:

PartialEq, Eq - Equality comparisons
Clone - Cloning support
Default - Default constructor (empty array)

Indexing:

Index<usize>, IndexMut<usize> - Array-style indexing (arr[0], arr[1] = 0xff)

§Implementation Notes

Feature Flags: Check Cargo.toml for hardening tiers. When implementing features, use individual feature flags (sec_harden_zeroize, sec_harden_const_time_ops, etc.) in #[cfg] attributes. Tiers (sec_basic_hardening, sec_enhanced_hardening, sec_maximum_hardening) are convenience bundles for users to activate multiple features at once.

Security Hardening: The trust module provides two mutually exclusive implementations:

Hardened mode (any sec_harden_* feature): Restricts operations to prevent data leakage
Insecure mode (default): Provides full Vec<u8> compatibility via Deref See the Security Hardening Guide below for detailed information.

Re-exports§

pub use core::errors;
pub use core::macros;
pub use core::model::ByteArray;

Modules§

core
security_guide: Security Hardening Guide

Macros§

try_bin: Macro for creating a ByteArray from a binary string literal (without 0b prefix).
try_bytes: Macro for creating a ByteArray from a string literal with automatic format detection.
try_hex: Macro for creating a ByteArray from a hexadecimal string literal (without 0x prefix).