Skip to main content

Module intern

Module intern 

Source
Expand description

String interning for DX-Machine

Provides zero-copy string deduplication to reduce serialized size by 50-90% for data with repeated strings (logs, configs, etc.).

§Architecture

┌─────────────────────────────────────────────────────────┐
│ Serialized Format                                       │
├─────────────────────────────────────────────────────────┤
│ Header (4 bytes) - FLAG_HAS_INTERN set                  │
├─────────────────────────────────────────────────────────┤
│ String Pool:                                            │
│   - Pool size: u32 (4 bytes)                            │
│   - String count: u32 (4 bytes)                         │
│   - Offsets: [u32; count] (4 * count bytes)             │
│   - Strings: concatenated UTF-8 data                    │
├─────────────────────────────────────────────────────────┤
│ RKYV Data (strings replaced with pool indices)          │
└─────────────────────────────────────────────────────────┘

Structs§

InternPool
String interning pool for deduplication
InterningDeserializer
Deserializer with string interning support
InterningSerializer
Serializer with string interning support

Enums§

InternError
Errors that can occur during string interning