Module v2 

APR Format Module (v2 APR\0) — sovereign leaf (issue #2231)

Implements the APR v2 container format with:

• 64-byte tensor alignment for zero-copy mmap
• LZ4 block compression (64KB blocks)
• JSON metadata section
• Multi-file sharding for 10B+ parameter models
• Single unified format (no versioning complexity)

Format Structure (APR)

┌─────────────────────────────────────────────────────────────┐
│ Header (64 bytes, 64-byte aligned)                          │
│   - Magic: "APR\0" (4 bytes) - ONE format, no versioning    │
│   - Version: major.minor (2 bytes)                          │
│   - Flags (2 bytes)                                         │
│   - Tensor count (4 bytes)                                  │
│   - Metadata offset (8 bytes)                               │
│   - Metadata size (4 bytes)                                 │
│   - Tensor index offset (8 bytes)                           │
│   - Data offset (8 bytes)                                   │
│   - Checksum (4 bytes)                                      │
│   - Reserved (20 bytes, zero-padded)                        │
├─────────────────────────────────────────────────────────────┤
│ JSON Metadata (variable, padded to 64-byte boundary)        │
├─────────────────────────────────────────────────────────────┤
│ Tensor Index (sorted by name, 64-byte aligned entries)      │
├─────────────────────────────────────────────────────────────┤
│ Tensor Data (each tensor 64-byte aligned)                   │
├─────────────────────────────────────────────────────────────┤
│ Footer Checksum (4 bytes)                                   │
└─────────────────────────────────────────────────────────────┘

Example

use apr_format::v2::{AprV2Header, AprV2Flags, MAGIC_V2, ALIGNMENT};

let header = AprV2Header::new();
assert_eq!(header.magic, MAGIC_V2);
assert!(header.is_valid());

Sovereignty (issue #2231)

This module contains ONLY the container I/O — pure bytes, shapes, and dtypes. It carries no ML/GPU/tokenizer dependency:

• CRC32 routes through the single crate::crc32::crc32.
• f16 conversion routes through crate::f16 (the IEEE-correct half crate), NOT trueno::f32_to_f16. See the f16 note in crate::f16.
• The dequantizing get_tensor_as_f32 accessor (which needs the GGUF Q4_K/Q6_K dequant + f32 physics) is severed from the leaf reader and re-attached in aprender-core as an extension trait (AprV2DequantExt). The leaf exposes the raw bytes via AprV2Reader::get_tensor_data and the typed-but-trivial AprV2Reader::get_f32_tensor (F32 dtype only).

Re-exports

pub use stamp::stamp_provenance_bytes;

pub use stamp::ProvenancePatch;

Modules

stamp

APR v2 provenance stamping — SHIP-009 full-discharge enabler.

Structs

AprV2Flags

APR v2 feature flags (16-bit for expanded feature set)

AprV2Header

APR file header (64 bytes)

AprV2Metadata

APR v2 JSON metadata section

AprV2Reader

APR v2 format reader (owns data - copies input)

AprV2ReaderRef

APR v2 format reader with zero-copy (borrows data - for mmap)

AprV2StreamingWriter

Streaming APR v2 writer — writes tensors to disk incrementally (realizar#136).

AprV2Writer

APR v2 format writer

ChatSpecialTokens

Special tokens for chat templates (CTA-04)

QuantizationMetadata

Quantization metadata

ShardInfo

Information about a single shard

ShardManifest

Shard manifest for multi-file models

ShardingMetadata

Sharding metadata for multi-file models

TensorIndexEntry

Tensor index entry (fixed size for efficient lookup)

Enums

TensorDType

Tensor data type for APR v2 format.

V2FormatError

APR v2 format error

Constants

ALIGNMENT

Tensor alignment in bytes (for zero-copy mmap)

HEADER_SIZE_V2

Header size in bytes (64-byte aligned)

LZ4_BLOCK_SIZE

LZ4 block size in bytes

MAGIC_V2

APR magic number: “APR\0” in ASCII (0x41505200) ONE format. No versioning. Period.

MAX_METADATA_SIZE

Maximum metadata size (16MB)

MAX_TENSOR_NAME_LEN

Maximum tensor name length

VERSION_V2

Format version 2.0

Functions

align_64

Align value up to 64-byte boundary

align_up

Align value up to the nearest multiple of alignment

is_aligned_64

Check if value is 64-byte aligned

padding_to_align

Calculate padding needed to reach alignment

required_file_len

The minimum file length the container’s own header + tensor index imply (issue #2612).