Crate erigon_seg 

Reader for the Erigon 3 seg state file format.

Erigon stores a snapshot of domain state (accounts, storage, code, …) as a triple of sibling files sharing one base name, e.g. v1.1-accounts.0-1024.{kv,bt,kvei}:

• .kv — the data: a seg-compressed stream of words. For a domain file the words alternate key, value, key, value, … and the keys are sorted.
• .bt — a B-tree index whose payload is an Elias-Fano array giving the .kv byte offset of every key, enabling an O(log n) point lookup. Its newer layout also carries the key at every M-th position, which narrows a lookup to a single M-key block before any decompression happens (see BtreeIndex::nodes).
• .kvei — an existence (bloom) filter: a negative accelerator. If it says a key is absent, the .bt search can be skipped entirely. It never reports a real key as absent (no false negatives), so it is safe to trust for the negative case.

This crate currently implements reading and querying. Writing and merging are planned as later additions.

A domain’s state usually spans several files covering successive step ranges, where a newer file overrides keys carried by an older one. KvStack wraps an ordered set of KvReaders and resolves point lookups newest-first so overrides win.

Quick start

use erigon_seg::{KvReader, Salt};

// Open a .kv and (if present) its sibling .bt / .kvei.
let mut r = KvReader::open("v1.1-accounts.0-1024.kv")?;

// The .kvei bloom needs the index salt to be useful; resolve it once.
r.enable_bloom(Salt::Find(8));

// Only worth setting when the data is large relative to RAM — see the method docs.
let _ = r.advise_random();

// Point lookup (bloom-accelerated if enabled, else B-tree binary search).
if let Some(value) = r.get(b"\x00\x01\x02")? {
    println!("value = {} bytes", value.len());
}

// Or scan every key/value pair sequentially.
for kv in r.iter() {
    let (key, value) = kv?;
    let _ = (key, value);
}

Format notes

The reader handles both released on-disk layouts:

• .kv: the legacy v0 header (body at offset 0) and the v1 header (a leading [version, feature-flags] pair, an optional page-compression byte, and optional out-of-band metadata).
• .bt: the legacy layout (Elias-Fano at offset 0) and the newer footer layout (a trailing erigon\0\0 magic locating the Elias-Fano section).
• .kvei: the holiman/bloomfilter layout (v02\n magic). The newer “fuse filter” layout is detected and skipped (lookups remain correct, just unaccelerated).

Structs

BtOptions

Options for building a .bt index.

BtreeIndex

A .bt index: the Elias-Fano offset array plus, when known, the B-tree fanout M and the di-node array used to narrow lookups.

DomainOptions

Options for DomainWriter.

DomainPaths

Paths written by DomainWriter::finish.

DomainWriter

Builds a domain file set from sorted (key, value) pairs.

EliasFano

A read-only view of an Elias-Fano monotone sequence.

ExistenceFilter

A .kvei existence filter.

Getter

A cursor over a Seg’s words. Cheap to create; not thread-safe, so each thread makes its own from a shared &Seg.

KvIter

Iterator over the (key, value) pairs of a KvReader, in stored order.

KvReader

A reader over one seg file set (.kv data, optional .bt index, optional .kvei existence filter).

KvStack

A stack of seg files for one domain, queried newest-first so overrides win.

KveiBuilder

Accumulates set bits for a .kvei bloom filter.

MergeOptions

Options for merge.

Nodes

The .bt di-node array: the key at every M-th position.

OpenOptions

Options controlling how a .kv is opened.

Seg

A seg-compressed file (.kv). Owns its mmap and dictionaries; create a Getter to read words.

SegWriter

Builds a seg .kv file from a sequence of words.

Enums

BtLayout

Which .bt on-disk layout to emit.

Error

Anything that can go wrong while opening or reading a seg file set.

FilterKind

What kind of filter a .kvei turned out to be.

Salt

How to obtain the .kvei index salt.

Constants

DEFAULT_BTREE_M

Default B-tree fanout (DefaultBtreeM), the number of keys per leaf.

Functions

build_bt

Build a .bt index for a .kv file, writing it to bt_path.

build_bt_from_seg

Build a .bt index from an already-open Seg.

build_kvei

Build a .kvei existence filter for a .kv file, writing it to out_path.

build_kvei_from_seg

Build a .kvei for every key in an already-open Seg, hashing with salt.

merge

Merge inputs (domain .kv paths) into out_kv, also building the sibling .bt and—if opts.domain.salt is set—.kvei.

murmur3_x64_128_h1

MurmurHash3 x64 128-bit, returning only the first 64-bit half (h1) — the value Erigon feeds to the existence filter as the hashed key.

salt_from_file

Read an Erigon salt file (salt-state.txt / salt-blocks.txt): a 4-byte big-endian u32. Returns None if the file is missing or too short.

Type Aliases

Result

The crate result type.