Module machine 

DX-Machine: Pure RKYV + LZ4 Implementation

Binary serialization format using RKYV for zero-copy deserialization and LZ4 for fast compression.

Format

• RKYV: Zero-copy binary serialization
• LZ4: Fast compression (enabled by default with compression feature)
• Arena-based: Flattened value storage to avoid recursive types

Features

• Serializable & deserializable with full round-trip support
• 41-74% smaller than LLM text format
• Zero-copy deserialization for maximum performance
• LZ4 compression for reduced wire size

Usage

use serializer::{llm_to_machine, machine_to_document};

// Convert LLM format to machine format
let machine = llm_to_machine(llm_text)?;

// Deserialize back to document
let doc = machine_to_document(&machine)?;

• DX-SIMD512: AVX-512 operations

These are preserved for future use but not currently active to keep the implementation simple and focused on pure RKYV.

Re-exports

pub use api::deserialize;

pub use api::deserialize_batch;

pub use api::serialize;

pub use api::serialize_batch;

pub use builder::DxMachineBuilder;

pub use cache::MACHINE_CACHE_HEADER_LEN;

pub use cache::MachineCacheCodec;

pub use cache::MachineCacheError;

pub use cache::MachineCacheKind;

pub use cache::MachineCacheMiss;

pub use cache::MachineCachePaths;

pub use cache::MachineCacheReceipt;

pub use cache::MachineCacheSchema;

pub use cache::MachineCacheSource;

pub use cache::MachineCacheWriteOptions;

pub use cache::access_typed_machine_cache;

pub use cache::paths_for_project_cache;

pub use cache::source_fingerprint;

pub use cache::write_typed_machine_cache;

pub use cache::json_source_to_typed_machine_cache;

pub use cache::toml_source_to_typed_machine_cache;

pub use format::DxFormat;

pub use format::FormatMode;

pub use format::detect_format;

pub use format::parse_auto;

pub use header::DxMachineHeader;

pub use header::FLAG_HAS_HEAP;

pub use header::FLAG_HAS_INTERN;

pub use header::FLAG_HAS_LENGTH_TABLE;

pub use header::FLAG_LITTLE_ENDIAN;

pub use serde_compat::from_bytes as from_bytes_serde;

pub use serde_compat::to_bytes as to_bytes_serde;

pub use slot::DxMachineSlot;

pub use slot::HEAP_MARKER;

pub use slot::INLINE_MARKER;

pub use slot::MAX_INLINE_SIZE;

pub use traits::DxMachineDeserialize;

pub use traits::DxMachineSerialize;

pub use types::DxMachineError;

pub use optimized_rkyv::CompressedRkyv;

pub use optimized_rkyv::OptimizedRkyv;

pub use compress::CompressionLevel;

pub use compress::DxCompressed;

pub use intern::InternError;

pub use intern::InternPool;

pub use intern::InterningDeserializer;

pub use intern::InterningSerializer;

Modules

api

High-level DX-Machine API with RKYV

blocking

Blocking I/O fallback implementation

builder

DX-Machine in-place serialization builder

cache

Typed .machine cache helpers for JSON/config/receipt/index read models.

compress

DX-Compress: Integrated LZ4 Streaming

footer

DX-Machine footer format (negative-offset)

format

Format detection and dual-mode support

header

DX-Machine header format

intern

String interning for DX-Machine

machine_types

RKYV-compatible types for machine format serialization

optimized_rkyv

Optimized RKYV wrapper using DX features

rkyv_compat

RKYV compatibility layer with DX-Machine ultra-performance enhancements

serde_compat

Serde-compatible serialization for DX-Machine format

simd

SIMD optimizations for DX-Machine

slot

DX-Machine unified slot format

traits

DX-Machine traits for serialization and deserialization

types

Machine format error types

Constants

MAGIC

DX-Machine magic bytes: 0x5A 0x44 (“ZD” little-endian)

SLOT_SIZE

Slot size in bytes (16 bytes for inline optimization)

VERSION

DX-Machine format version

Traits

AsyncFileIO

Unified file I/O trait used by machine-format optimization helpers.