Struct VersionedEmbrFS 

pub struct VersionedEmbrFS {
    pub chunk_store: VersionedChunkStore,
    pub corrections: VersionedCorrectionStore,
    pub manifest: VersionedManifest,
    /* private fields */
}

A mutable, versioned filesystem backed by holographic engrams

VersionedEmbrFS provides read-write operations with optimistic locking, enabling concurrent access while maintaining consistency and bit-perfect reconstruction guarantees.

Fields

chunk_store: VersionedChunkStore

Versioned chunk store (chunk_id → encoded SparseVec)

corrections: VersionedCorrectionStore

Versioned corrections for bit-perfect reconstruction

manifest: VersionedManifest

Versioned manifest (file metadata)

Implementations

impl VersionedEmbrFS

pub fn new() -> Self

Create a new empty mutable filesystem (legacy mode)

pub fn new_holographic() -> Self

Create a new mutable filesystem with holographic mode enabled

In holographic mode, data is encoded using ReversibleVSAEncoder which achieves ~94% uncorrected accuracy through position-aware VSA binding. Only ~6% of bytes need correction, resulting in <10% correction overhead.

pub fn with_config(config: ReversibleVSAConfig) -> Self

Create a new mutable filesystem with custom VSA configuration

pub fn with_config_and_profiler( config: ReversibleVSAConfig, profiler: CompressionProfiler, ) -> Self

Create a new mutable filesystem with custom VSA configuration and compression profiler

pub fn enable_holographic_mode(&mut self)

Enable holographic mode on an existing filesystem

New writes will use ReversibleVSAEncoder for encoding with ~94% accuracy.

pub fn reversible_encoder(&self) -> &Arc<RwLock<ReversibleVSAEncoder>>

Get a reference to the reversible encoder

pub fn is_holographic(&self) -> bool

Check if holographic mode is enabled

pub fn codebook(&self) -> &Arc<RwLock<Codebook>>

Get a reference to the codebook

pub fn profiler(&self) -> &CompressionProfiler

Get the compression profiler

pub fn version(&self) -> u64

Get the current global version

pub fn read_file(&self, path: &str) -> Result<(Vec<u8>, u64), EmbrFSError>

Read a file’s contents

Returns the file data and the file entry version at read time. The version can be used for optimistic locking on subsequent writes.

Example

let fs = VersionedEmbrFS::new();
let (data, version) = fs.read_file("example.txt")?;
println!("Read {} bytes at version {}", data.len(), version);

pub fn read_range( &self, path: &str, offset: usize, length: usize, ) -> Result<(Vec<u8>, u64), EmbrFSError>

Read a specific byte range from a file

This method enables efficient partial file reads by only decoding the chunks needed to satisfy the requested byte range. When the file has a chunk offset index, chunks are located in O(log n) time.

Arguments

• path - The file path within the engram
• offset - The starting byte offset to read from
• length - The number of bytes to read

Returns

A tuple of (data, version) where data is the requested byte range. If the range extends beyond the file, only available bytes are returned.

Performance

• With offset index: O(log n + k) where k = number of chunks in range
• Without offset index: O(n) where n = total chunks (must scan all)

Example

let fs = VersionedEmbrFS::new();
// Read bytes 1000-1999 from a file
let (data, version) = fs.read_range("large_file.bin", 1000, 1000)?;

pub fn apply_delta(&self, path: &str, delta: &Delta) -> Result<u64, EmbrFSError>

Apply a delta operation to a file

Delta encoding allows efficient modification of files without full re-encoding. For non-shifting operations (ByteReplace, RangeReplace), only affected chunks are re-encoded, providing significant speedup for large files.

Performance

Operation	Full Re-encode	Delta	Speedup
1 byte in 1MB file	~90ms	~1ms	~90x
10 bytes in 1MB file	~90ms	~10ms	~9x
Append 1KB to 1MB file	~90ms	~1ms	~90x

Arguments

• path - Path to the file to modify
• delta - The delta operation to apply

Returns

The new file version after applying the delta

Example

let fs = VersionedEmbrFS::new();

// Create a file
let version = fs.write_file("data.txt", b"Hello World", None)?;

// Replace 'W' with 'w' at position 6
let delta = Delta::with_version(
    DeltaType::ByteReplace {
        offset: 6,
        old_value: b'W',
        new_value: b'w',
    },
    version,
);
let new_version = fs.apply_delta("data.txt", &delta)?;

// File now contains "Hello world"

pub fn encode_chunk(&self, data: &[u8], path: Option<&str>) -> SparseVec

Encode chunk data using appropriate encoder based on mode.

When holographic mode is enabled, uses ReversibleVSAEncoder for ~94% uncorrected accuracy. Otherwise uses legacy SparseVec::encode_data().

pub fn decode_chunk( &self, vec: &SparseVec, path: Option<&str>, original_size: usize, ) -> Vec<u8>

Decode chunk data using appropriate decoder based on mode.

Must match the encoding method: use ReversibleVSAEncoder::decode for holographic mode, SparseVec::decode_data for legacy mode.

pub fn encode_chunk_with_format( &self, data: &[u8], path: Option<&str>, encoding_format: Option<u8>, ) -> SparseVec

Encode chunk data using an explicit encoding format.

Use this when encoding chunks for existing files that have a specific encoding_format stored in their manifest entry, to preserve consistency.

pub fn decode_chunk_with_format( &self, vec: &SparseVec, path: Option<&str>, original_size: usize, encoding_format: Option<u8>, ) -> Vec<u8>

Decode chunk data using an explicit encoding format.

Use this when decoding chunks for existing files that have a specific encoding_format stored in their manifest entry, to ensure correct decoding.

pub fn write_file( &self, path: &str, data: &[u8], expected_version: Option<u64>, ) -> Result<u64, EmbrFSError>

Write a file’s contents

If expected_version is provided, performs optimistic locking - the write will fail with VersionMismatch if the file has been modified since the version was read.

Example

let fs = VersionedEmbrFS::new();

// Create new file
let version = fs.write_file("new.txt", b"content", None)?;

// Update with version check
let new_version = fs.write_file("new.txt", b"updated", Some(version))?;

pub fn write_file_compressed( &self, path: &str, data: &[u8], expected_version: Option<u64>, ) -> Result<u64, EmbrFSError>

Write a file’s contents with path-based automatic compression

Uses the compression profiler to automatically select the appropriate compression codec based on the file path. For example, config files in /etc get fast LZ4 compression, kernel images get maximum zstd compression, and pre-compressed media files skip compression entirely.

If expected_version is provided, performs optimistic locking - the write will fail with VersionMismatch if the file has been modified since the version was read.

Example

let fs = VersionedEmbrFS::new();

// Config file gets fast LZ4 compression automatically
let version = fs.write_file_compressed("/etc/nginx.conf", b"worker_processes 4;", None)?;

// Kernel image gets maximum zstd compression
let kernel_data = std::fs::read("/boot/vmlinuz")?;
fs.write_file_compressed("/boot/vmlinuz", &kernel_data, None)?;

pub fn delete_file( &self, path: &str, expected_version: u64, ) -> Result<(), EmbrFSError>

Delete a file (soft delete)

The file is marked as deleted but its chunks remain in the engram until compaction. Requires the current file version for optimistic locking.

pub fn list_files(&self) -> Vec<String>

List all non-deleted files

pub fn exists(&self, path: &str) -> bool

Check if a file exists

pub fn stats(&self) -> FilesystemStats

Get filesystem statistics

pub fn write_file_holographic( &self, path: &str, data: &[u8], expected_version: Option<u64>, ) -> Result<u64, EmbrFSError>

Write a file using holographic encoding via ReversibleVSAEncoder

This is the TRUE holographic storage method achieving ~94% uncorrected accuracy:

1. Encode data using ReversibleVSAEncoder.encode_chunked() (position-aware binding)
2. Store encoded SparseVecs in chunk_store (one per REVERSIBLE_CHUNK_SIZE bytes)
3. Decode to verify and compute corrections for bit-perfect reconstruction
4. Store only sparse corrections (~6% of bytes need correction)

The position-aware binding ensures each byte at each position has a unique representation that can be retrieved via unbinding.

pub fn read_file_holographic( &self, path: &str, ) -> Result<(Vec<u8>, u64), EmbrFSError>

Read a file using holographic decoding via ReversibleVSAEncoder

This reverses the holographic encoding:

1. Load all SparseVecs from chunk_store
2. Use ReversibleVSAEncoder.decode_chunked() to reconstruct data
3. Apply per-chunk corrections for bit-perfect result

Supports both legacy (format 0) and new (format 1) encoding formats.

pub fn config(&self) -> &ReversibleVSAConfig

Get the VSA configuration

pub fn stream_decode( &self, path: &str, ) -> Result<StreamingDecoder<'_>, EmbrFSError>

Create a streaming decoder for memory-efficient file reading

Returns a StreamingDecoder that decodes chunks on-demand, keeping memory usage bounded regardless of file size. Use this for large files where loading the entire file into memory is impractical.

Example

let fs = VersionedEmbrFS::new();

// Create streaming decoder
let mut decoder = fs.stream_decode("large_file.bin")?;

// Read in chunks without loading entire file
let mut buffer = vec![0u8; 4096];
while let Ok(n) = decoder.read(&mut buffer) {
    if n == 0 { break; }
    // Process buffer[..n]
}

pub fn stream_decode_range( &self, path: &str, offset: usize, max_bytes: Option<usize>, ) -> Result<StreamingDecoder<'_>, EmbrFSError>

Create a streaming decoder with custom options

Allows setting starting offset and maximum bytes to read for partial decoding.

Example

let fs = VersionedEmbrFS::new();

// Read bytes 1000-2000 from a file
let decoder = fs.stream_decode_range("large_file.bin", 1000, Some(1000))?;

pub fn allocate_chunk_id(&self) -> ChunkId

Allocate a new unique chunk ID (public for streaming API)

pub fn bundle_chunks_to_root_streaming( &self, chunk_ids: &[ChunkId], ) -> Result<(), EmbrFSError>

Bundle chunks into root - streaming variant that doesn’t retry on mismatch

For streaming ingestion, we bundle progressively without requiring atomicity since we’re building up the root from scratch.

Trait Implementations

impl Default for VersionedEmbrFS

fn default() -> Self

Returns the “default value” for a type.