Module format_migration 

Block Format Migration - Backwards Compatible Format Versioning

This module provides format versioning for SochDB’s block storage, enabling backwards-compatible upgrades without data loss.

Design

┌─────────────────────────────────────────────────────────────┐
│                    Block Format Versions                     │
│                                                             │
│  v1: Original format (17-byte header + data)                │
│  v2: Extended header (21-byte: +format version, +flags)     │
│  v3: Future format with additional metadata                 │
│                                                             │
│  Detection Strategy:                                        │
│  - v1: magic="TBLK", byte[17] = data start                  │
│  - v2: magic="TBL2", byte[5] = format version               │
│  - All versions share first 4 bytes for magic detection     │
└─────────────────────────────────────────────────────────────┘

Format Detection

The reader auto-detects format version by inspecting magic bytes:

• TBLK = Format v1 (original 17-byte header)
• TBL2 = Format v2 (extended 21-byte header)

Migration Path

Blocks are migrated lazily on read:

1. Read block with auto-detection
2. If format < current, upgrade in memory
3. Optionally rewrite upgraded block during compaction

Structs

BlockFlags

Block flags for v2 format

FormatMigrator

Block format migrator for batch migrations

MigratableBlock

Complete block with header and data

MigrationStats

Block format migration statistics

V1Header

V1 block header (original format, 17 bytes)

V2Header

V2 block header (extended format, 21 bytes)

Enums

BlockHeader

Version-agnostic block header

FormatVersion

Block format version