Module spine 

The Spine: every layer header of the stream in one shared mmap file.

spine.bin is Slot::WalStart + 8 pages of 16 KiB. Slot 0 carries the superblock, slots 1..=11 carry one header page per layer, slot 12 opens the write-ahead region. A process that owns the stream writes a header when its aggregates change; the MCP/CLI readers and the live-viewer process read those pages without going through the writer at all.

The publish protocol, in one paragraph

A page write is a 16 KiB copy_from_slice. It is not an atomic operation, and Mutex<MmapMut> only serialises handles of one process: two processes mapping the same file have two mutexes, so they have no happens-before relation and a reader can copy a half-written page or pair a new stream header with an older one. That tearing is what the seqlock in this file exists to remove, and nothing else here claims to.

The superblock page reserves SEQ_SLOT as its publish counter, exactly the way crate::consciousness_seqlock::MmapSnapshot reserves sequence: AtomicU64 at offset 0 of its own file. A publisher makes the counter odd, writes the bracketed payload, then stores the next even value with Ordering::Release; a reader loads with Ordering::Acquire, spins while odd, copies, and retries if the counter moved. The reader therefore never returns a payload from inside a publish window, and it does not need the mutex to find that out.

What the counter does and does not cover

Every header write participates in the counter. Spine::begin_publish holds one mapping mutex and one publisher lock across a read-modify-write batch; its first write opens the window, and commit (or drop) closes it. Spine::read_store_snapshot validates all eleven header prefixes and the superblock in one window. Spine::read_snapshot is the cheaper stream-only view. Separate legacy read_* calls do not form a validated snapshot.

A window guarantees a coherent copy, not rollback or durability. Dropping a batch during unwinding publishes whatever was written before the panic; crash recovery likewise cannot reconstruct the pre-crash state. Aggregates refreshed less frequently than the stream may legitimately be older.

Who is allowed to be inside a window

A seqlock only helps if at most one writer is in a window at a time; two overlapping windows can leave a reader validating a counter that neither writer owns. So publishing also takes flock(LOCK_EX) on spine.publish (see Spine::begin_publish), which is the piece shared pages cannot provide: mutual exclusion between publishers in different processes.

A second lock file answers a different question. spine.lock is held shared for the whole life of a writer handle, and exclusively by first-time layout and by Spine::resize, because remapping the file invalidates the pages every other handle is reading. It cannot double as the publish lock: a handle holding it shared would deadlock against itself trying to take it exclusively for one tick.

flock is advisory, is not available on Windows, and does not exist on every filesystem; all of those degrade to “publishing is serialised per handle only”. That is a real weakening of the guarantee, so it is reported by Spine::publish_lock_is_advisory instead of being hidden.

Lock order is mapping mutex, then publisher lock. flock belongs to the open file description, not the thread: taking it before the mutex would let a second thread on the same handle unlock the first thread’s publish. The advisory lock is released before the mapping mutex on exit.

Structs

ArchetypeHeader

AttentionHeader

CacheHeader

EmotionHeader

GraphHeader

HebbianHeader

MerkleHeader

MetaHeader

MirrorHeader

Publish

One exclusive read-modify-write batch. Do not call methods on the owning Spine while this guard lives: its mapping mutex is non-reentrant. Use the guard’s read_* / write_* methods instead. Reads alone do not advance the counter; the first write opens the window and commit/drop closes it.

PublishGuard

One publisher’s turn. Drop releases the advisory lock, so a publish that panics part-way through cannot leave the store un-writable.

PublishSnapshot

One published snapshot: the stream header plus the superblock payload that was in the file at the same moment — the two things the seqlock brackets. Their agreement is the guarantee; see Spine::read_snapshot.

ResonanceHeader

Spine

Spine file handle. Clone-like sharing is by Arc inside crate::consciousness_stream; a second process opens its own handle over the same file and coordinates through the seqlock.

StoreSnapshot

All header pages and the publish record copied under one counter value. Layer aggregates can be older than the stream, but none was copied midway through a write. Use a single Publish batch to update related headers.

StreamHeader

Superblock

Superblock, as it lives in the first SUPERBLOCK_BYTES bytes of page 0.

Enums

Slot

Constants

ATTENTION_LAYERS

HEADER_SIZE

MAGIC

N_LAYERS

PAGE_SIZE

PUBLISH_LOCK_FILE_NAME

Name of the file whose advisory lock serialises publishers.

SEQ_SLOT

Byte offset of the seqlock publish counter inside the superblock page (page 0, so this is also the file offset). 8-byte aligned by construction because mappings start on a page boundary.

SNAPSHOT_MAX_RETRIES

Read attempts on a publish window that keeps moving under us before the reader gives up and reports None. Spins for the first 64, then yields, so the budget is a measure of patience rather than of CPU burned competing with the writer. It is large because an attempt costs a lock hand-over when a reader shares its handle with other readers: measured against three threads hammering one file, a 1024-attempt budget gave up on 0.09% of reads even though the writer’s window is never longer than a microsecond, purely from losing the lock. At this budget it gives up on none.

SNAPSHOT_MAX_WAIT_MS

…and the shortest time it will keep trying, in milliseconds. An attempt is cheap, so on an oversubscribed machine the count can run out while the reader has merely been off-CPU; the deadline is what distinguishes “the writer is relentless” from “I was not scheduled”. Measured on a 2-core box with three load tests and a spawned writer running at once: count alone gave up on about 1 read in 60 000, this gives up on none.

SNAPSHOT_PREFIX_BYTES

Bytes copied from each header page by a validated snapshot. The largest encoded header is Emotion (108 bytes); no page padding needs to be read. This is a format bound, not a measured performance claim.

SUPERBLOCK_BYTES

Bytes of the superblock page that are part of the published payload: the fixed-size header fields, padded out so the payload ends where the counter begins. Spine::write_slot accepts a whole page for this slot but publishes only these bytes, precisely so that a page-sized write cannot reach into SEQ_SLOT and reset the counter under a reader.

SUPERBLOCK_VERSION

Superblock format version.

WRITER_LOCK_FILE_NAME

Name of the file whose advisory lock marks live writers for resize.

Functions

format_summary

Human-readable dump of the spine.