Module omen

Module omen 

Source
Expand description

.omen single-file storage format for OmenDB

Layout:

┌─────────────────────────────────────────────────────────────┐
│ HEADER (4KB, page 0)                                        │
├─────────────────────────────────────────────────────────────┤
│ VECTOR SECTION (page-aligned, mmap)                         │
├─────────────────────────────────────────────────────────────┤
│ GRAPH SECTION (page-aligned, mmap)                          │
├─────────────────────────────────────────────────────────────┤
│ METADATA SECTION                                            │
├─────────────────────────────────────────────────────────────┤
│ WAL SECTION (append-only, at end)                           │
└─────────────────────────────────────────────────────────────┘

Structs§

GraphSection
Graph section - stores HNSW neighbor lists
MetadataIndex
Metadata index - collection of field indexes
OmenFile
OmenFile - single-file vector database
OmenHeader
.omen file header
SectionEntry
Section entry in header (24 bytes)
VectorSection
Vector section - contiguous array of vectors
Wal
Write-Ahead Log
WalEntry
WAL entry (header + data)

Enums§

FieldIndex
Field index - wraps different index types
Filter
Filter expressions
FilterValue
Filter value types
Metric
Distance metric for similarity search (user-facing API type).
SectionType
Section types
WalEntryType
WAL entry types

Constants§

HEADER_SIZE
Header size (4KB, one page)
MAGIC
Magic bytes: “OMEN”
PAGE_SIZE
Page size for alignment (8KB optimal for NVMe)
VERSION_MAJOR
Current format version
VERSION_MINOR

Functions§

align_to_page
Align a value to page boundary