Module schema_bridge

sochdb_core

Module schema_bridge 

Source
Expand description

Schema Bridge - TOON to Columnar Mapping

This module provides bidirectional mapping between TOON document format and columnar storage format for efficient analytical queries.

§Design

┌─────────────────────────────────────────────────────────────────┐
│                      Schema Bridge                               │
│                                                                 │
│  TOON Document Format          Columnar Format                  │
│  ┌──────────────────┐          ┌──────────────────┐            │
│  │ users[3]{id,name}│          │ Column: id       │            │
│  │ 1,Alice          │   ←───→  │ [1, 2, 3]        │            │
│  │ 2,Bob            │          │                  │            │
│  │ 3,Carol          │          │ Column: name     │            │
│  └──────────────────┘          │ ["Alice","Bob",  │            │
│                                │  "Carol"]        │            │
│                                └──────────────────┘            │
│                                                                 │
│  Mapping Strategy:                                              │
│  • Primitive fields → Direct column mapping                     │
│  • Nested objects → Flattened with dot notation                │
│  • Arrays → Repeated columns with indices                       │
│  • Nulls → Validity bitmap                                      │
└─────────────────────────────────────────────────────────────────┘

§Benefits

  • Vectorized Operations: SIMD-friendly columnar data
  • Compression: Better compression ratios for homogeneous data
  • Cache Efficiency: Access only needed columns
  • Predicate Pushdown: Filter before materialization

Structs§

Column
Columnar storage for a single column
ColumnMapping
Mapping between TOON schema and columnar schema
ColumnarStats
Statistics for columnar operations
ColumnarTable
Columnar representation of a TOON table
SchemaBridge
Schema bridge for converting between TOON and columnar formats

Enums§

ColumnData
Column data storage
ColumnType
Column data type for columnar storage