# OxiCode
A modern binary serialization library for Rust - the successor to bincode.
[](https://github.com/cool-japan/oxicode/actions)
[](https://crates.io/crates/oxicode)
[](https://opensource.org/licenses/MIT)
## About
OxiCode is a compact encoder/decoder pair that uses a binary zero-fluff encoding scheme. The size of the encoded object will be the same or smaller than the size that the object takes up in memory in a running Rust program.
This project serves as the spiritual successor to [bincode](https://github.com/bincode-org/bincode), maintaining **100% binary compatibility** while introducing modern improvements and advanced features that make it 150% better.
## Features
### Core Features (100% Bincode Compatible)
- **Compact encoding**: Minimal overhead in serialized format
- **Fast**: Optimized for performance with zero-copy operations where possible
- **Flexible**: Support for various encoding configurations
- **Safe**: Strict no-unwrap policy, comprehensive error handling
- **Modern**: Built with latest Rust practices and 2021 edition features
- **no_std support**: Works in embedded and resource-constrained environments
- **Bincode compatibility**: 100% binary format compatibility with bincode 2.0
### 150% Enhancement Features (Beyond Bincode)
- **⚡ SIMD Optimization**: Hardware-accelerated array encoding (2-4x speedup)
- **🗜️ Compression**: LZ4 (fast) and Zstd (better ratio) support
- **📦 Schema Evolution**: Version tracking and automatic migration
- **🌊 Streaming**: Chunked encoding/decoding for large datasets
- **⏱️ Async Streaming**: Non-blocking async I/O with tokio
- **✅ Validation**: Constraint-based validation middleware
See [Feature Comparison](#feature-comparison) below for detailed breakdown.
## Why OxiCode?
While bincode has served the Rust community well, OxiCode brings:
1. **100% Binary Compatibility**: Drop-in replacement with identical binary format
2. **Modern Rust practices**: Built from the ground up with Rust 2021 edition
3. **Safety first**: Strict no-unwrap policy throughout the codebase
4. **Better error handling**: More informative error messages and comprehensive error types
5. **Advanced features**: SIMD, compression, streaming, async, validation - features bincode lacks
6. **Active maintenance**: Dedicated to long-term support and evolution
## Installation
Add this to your `Cargo.toml`:
```toml
[dependencies]
oxicode = "0.1"
# With serde support (for serde::Serialize/Deserialize types)
oxicode = { version = "0.1", features = ["serde"] }
# Optional features
oxicode = { version = "0.1", features = ["simd", "compression", "async-tokio"] }
```
### Feature Flags
```toml
default = ["std", "derive"]
std = ["alloc"] # Standard library support
alloc = [] # Heap allocations (for no_std + alloc)
derive = [] # Derive macros for Encode/Decode
serde = [] # Serde integration (optional)
simd = [] # SIMD-accelerated array encoding
compression-lz4 = [] # LZ4 compression (fast)
compression-zstd = [] # Zstd compression (better ratio)
compression = ["compression-lz4"] # Default compression
async-tokio = ["tokio"] # Async streaming with tokio
async-io = ["futures-io"] # Generic async IO traits
```
## Quick Start
```rust
use oxicode::{Encode, Decode};
#[derive(Encode, Decode, PartialEq, Debug)]
struct Point {
x: f32,
y: f32,
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let point = Point { x: 1.0, y: 2.0 };
// Encode to bytes
let encoded = oxicode::encode_to_vec(&point)?;
// Decode from bytes
let (decoded, _): (Point, _) = oxicode::decode_from_slice(&encoded)?;
assert_eq!(point, decoded);
Ok(())
}
```
## Using with Serde
OxiCode provides optional serde integration for types that implement `serde::Serialize` and `serde::Deserialize`:
```rust
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug, PartialEq)]
struct Person {
name: String,
age: u32,
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let person = Person {
name: "Alice".to_string(),
age: 30,
};
// Encode using serde integration
let encoded = oxicode::serde::encode_to_vec(&person, oxicode::config::standard())?;
// Decode using serde integration
let (decoded, _): (Person, _) = oxicode::serde::decode_from_slice(&encoded, oxicode::config::standard())?;
assert_eq!(person.name, decoded.name);
assert_eq!(person.age, decoded.age);
Ok(())
}
```
**Enable serde feature in Cargo.toml:**
```toml
[dependencies]
oxicode = { version = "0.1", features = ["serde"] }
serde = { version = "1.0", features = ["derive"] }
```
## Configuration
OxiCode supports various encoding configurations:
```rust
use oxicode::config;
// Standard configuration (default): little-endian + varint
let cfg = config::standard();
// Legacy bincode 1.0-compatible: little-endian + fixed-int
let cfg = config::legacy();
// Custom configuration
let cfg = config::standard()
.with_big_endian()
.with_fixed_int_encoding()
.with_limit::<1048576>(); // 1MB limit
// Use with encoding/decoding
let bytes = oxicode::encode_to_vec_with_config(&value, cfg)?;
let (decoded, _) = oxicode::decode_from_slice_with_config(&bytes, cfg)?;
```
## Advanced Features
### SIMD-Accelerated Arrays
Hardware acceleration for large array operations (2-4x speedup):
```rust
use oxicode::{Encode, Decode};
#[derive(Encode, Decode)]
struct LargeDataset {
readings: Vec<f64>, // SIMD-accelerated when feature enabled
}
// Enable with features = ["simd"]
// Auto-detects CPU capabilities (SSE2, AVX2, AVX-512)
```
See `examples/simd_arrays.rs` for detailed usage.
### Compression
Reduce size with LZ4 or Zstd compression:
```rust
use oxicode::compression::{CompressedEncoder, CompressedDecoder, CompressionType};
// LZ4 - fast compression
let mut encoder = CompressedEncoder::new(writer, CompressionType::Lz4)?;
value.encode(&mut encoder)?;
// Zstd - better compression ratio
let mut encoder = CompressedEncoder::new(writer, CompressionType::Zstd(10))?;
value.encode(&mut encoder)?;
```
See `examples/compression.rs` for detailed usage.
### Streaming Serialization
Process large datasets incrementally:
```rust
use oxicode::streaming::{StreamingEncoder, StreamingDecoder};
// Encode items one at a time
let mut encoder = StreamingEncoder::new(writer, config)?;
for item in large_dataset {
encoder.write_item(&item)?;
}
encoder.finish()?;
// Decode items incrementally
let mut decoder = StreamingDecoder::new(reader, config)?;
while let Some(item) = decoder.read_item::<MyType>()? {
process(item);
}
```
See `examples/streaming.rs` for detailed usage.
### Async Streaming
Non-blocking async I/O with tokio:
```rust
use oxicode::streaming::AsyncStreamingEncoder;
// Async encoding
let mut encoder = AsyncStreamingEncoder::new(writer, config);
for item in dataset {
encoder.write_item(&item).await?;
}
let writer = encoder.finish().await?;
```
See `examples/async_streaming.rs` for detailed usage.
### Validation Middleware
Validate data during decoding:
```rust
use oxicode::validation::{Validator, Constraints};
// Create validator with constraints
let mut validator = Validator::new();
validator.add_constraint("name", Constraints::max_len(100));
validator.add_constraint("age", Constraints::range(Some(0), Some(120)));
// Validate decoded data
validator.validate(&person)?;
```
See `examples/validation.rs` for detailed usage.
### Schema Evolution
Version your data formats and migrate gracefully:
```rust
use oxicode::versioning::{Version, VersionedEncoder};
let version = Version::new(1, 0, 0);
let mut encoder = VersionedEncoder::new(writer, version, config)?;
value.encode(&mut encoder)?;
// Decoder automatically validates version compatibility
```
See `examples/versioning.rs` for detailed usage.
## Migration from bincode
OxiCode is 100% binary-compatible with bincode. Migration is straightforward:
```rust
// Before (bincode 2.0)
use bincode::{Encode, Decode, config};
let bytes = bincode::encode_to_vec(&value, config::standard())?;
let (decoded, _) = bincode::decode_from_slice(&bytes, config::standard())?;
// After (oxicode) - same API!
use oxicode::{Encode, Decode, config};
let bytes = oxicode::encode_to_vec(&value, config::standard())?;
let (decoded, _) = oxicode::decode_from_slice(&bytes, config::standard())?;
```
**Binary data is 100% compatible** - you can mix libraries:
- Data encoded with bincode can be decoded with oxicode ✓
- Data encoded with oxicode can be decoded with bincode ✓
For detailed migration guide, see [MIGRATION.md](MIGRATION.md).
## Feature Comparison
| Binary Compatibility | ✓ | ✗ | ✗ | ✗ | ✓ |
| Zero-copy | ✗ | ✓ | ✗ | ✗ | ✓ |
| no_std | ✓ | ✓ | ✓ | ✓ | ✓ |
| SIMD Optimization | ✗ | ✗ | ✗ | ✗ | ✓ |
| Compression | ✗ | ✗ | ✗ | ✗ | ✓ |
| Async Streaming | ✗ | ✗ | ✗ | ✗ | ✓ |
| Validation | ✗ | ✗ | ✗ | ✗ | ✓ |
| Schema Evolution | ✗ | ✗ | ✗ | ✗ | ✓ |
| Varint Encoding | ✓ | ✗ | ✓ | ✗ | ✓ |
## Project Status
**🎯 Version 0.1.0 - Production Ready**
All core features and enhancements complete. See [CHANGELOG.md](CHANGELOG.md) for details.
**Statistics** (as of 2025-12-28):
- **Lines of Code**: 10,860 (Rust source, excluding tests)
- **Files**: 61 Rust files
- **Test Coverage**: 211 tests passing (100% pass rate)
- 18 binary compatibility tests (100% byte-for-byte identical to bincode)
- 193+ feature and integration tests
- **Type Coverage**: 112+ types with full Encode/Decode support
- **Binary Compatibility**: 100% verified through cross-library testing
- **Code Quality**: ✓ Zero unwrap(), ✓ Zero warnings, ✓ All files < 2000 lines
## Project Structure
This is a workspace with the following crates:
- `oxicode`: Main library crate
- `oxicode_derive`: Procedural macros for deriving Encode/Decode
- `oxicode_compatibility`: Compatibility tests and bincode interop
## Development Principles
OxiCode follows strict development principles:
- **No warnings policy**: All code must compile without warnings
- **No unwrap policy**: All error cases must be properly handled
- **Latest crates policy**: Use latest versions of dependencies
- **Workspace policy**: Proper workspace structure with shared dependencies
- **Refactoring policy**: Keep individual files under 2000 lines
## Performance
OxiCode is designed for performance:
- **SIMD acceleration**: 2-4x speedup for large arrays (with `simd` feature)
- **Zero-copy deserialization**: Where possible
- **Efficient varint encoding**: For integers
- **Minimal allocations**: During encoding/decoding
- **Benchmark suite**: Included in `benches/`
Run benchmarks:
```bash
cargo bench
```
## Testing
```bash
# Run all tests
cargo nextest run --all-features
# Run specific feature tests
cargo test --features simd
cargo test --features compression
cargo test --features async-tokio
# Run with no-std
cargo test --no-default-features --features alloc
```
## Examples
The `examples/` directory contains comprehensive examples:
- `basic_usage.rs` - Simple encoding/decoding
- `configuration.rs` - Configuration options
- `zero_copy.rs` - Zero-copy deserialization
- `simd_arrays.rs` - SIMD-accelerated arrays
- `compression.rs` - LZ4 and Zstd compression
- `streaming.rs` - Chunked streaming
- `async_streaming.rs` - Async tokio streaming
- `validation.rs` - Validation middleware
- `versioning.rs` - Schema evolution
Run examples:
```bash
cargo run --example basic_usage
cargo run --example simd_arrays --features simd
cargo run --example compression --features compression
cargo run --example async_streaming --features async-tokio
```
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
## License
Licensed under the MIT license. See [LICENSE](LICENSE.md) for details.
## Acknowledgments
This project builds upon the excellent work done by the bincode team and community. We're grateful for their contributions to the Rust ecosystem.
## Related Projects
- [SciRS2](https://github.com/cool-japan/scirs) - Scientific computing library
- [NumRS2](https://github.com/cool-japan/numrs) - Numerical computing library
- [ToRSh](https://github.com/cool-japan/torsh) - PyTorch-like tensor library
- [OxiRS](https://github.com/cool-japan/oxirs) - RDF and SPARQL library
- [QuantRS2](https://github.com/cool-japan/quantrs) - Quantum computing library