Skip to main content

Module protocol_direct_file

Module protocol_direct_file 

Source
Expand description

DirectFileRing: non-mmap positioned-I/O ring that bypasses the OS page cache (cross-platform).

Where the substrate’s other ring primitives use mmap to share memory with peers, DirectFileRing opens a file in unbuffered mode and reads/writes via positioned I/O. The page cache is bypassed: every write goes directly to the underlying block device, every read comes directly from the device. Useful when the substrate IS the buffer (the caller does its own caching and does not want the kernel double-buffering) - common in database storage engines.

§Cross-platform mechanism

The unbuffered-I/O surface differs per OS; only that surface is gated, the ring layout and coordination are shared:

  • Unix: O_DIRECT open flag, pwrite(2) / pread(2), posix_memalign(3) aligned buffers.
  • Windows: FILE_FLAG_NO_BUFFERING + FILE_FLAG_WRITE_THROUGH open flags, WriteFile / ReadFile with an OVERLAPPED offset for positioned I/O, VirtualAlloc page-aligned buffers.

§Alignment

Both O_DIRECT and FILE_FLAG_NO_BUFFERING require that buffer addresses, file offsets, and transfer lengths be aligned to the device’s logical block / sector size (typically 512 or 4096 bytes). This primitive fixes the slot size at 4096 bytes and uses page-aligned buffers, which satisfies both.

§Coordination

Head/tail counters live in a SEPARATE small MMF (SharedAtomicU64) because writing them through the unbuffered data path would defeat their purpose (atomic visibility across processes). The data file holds payload slots only; the control files hold head + tail. The data is device-resident (write-through / O_DIRECT), so an independent reader process sees the producer’s writes once it observes the head counter advance.

Structs§

DirectFileRing
Non-mmap positioned-I/O ring with page-cache bypass.

Enums§

DirectFileError
Errors DirectFileRing operations can return.

Constants§

DIRECT_FILE_SLOT_SIZE
Fixed slot size matching the most common modern 4K-sector alignment. All positioned reads/writes are exactly this size.