FWOB

FWOB is a Rust implementation of the Fixed-Width Ordered Binary format family.

The project has two compatibility goals:

1. Fully support FWOB v1 files produced by the original C# library.
2. Provide FWOB v2, a fixed-page compressed format for high-performance random access, range queries, and bulk append workloads.

FWOB v2 keeps page addresses arithmetic while allowing each page to contain a variable number of fixed-width frames. A page is a fixed-size on-disk container with a 64-byte header, compressed payload, and zero padding.

Workspace

• fwob-core: shared schema, frame, key, reader/writer handles, service traits, and error types.
• fwob-derive: derive macro for strongly typed fixed-width frames.
• fwob-v1: FWOB v1 reader, writer, verifier, and compatibility tests.
• fwob-v2: compressed fixed-page FWOB v2 reader and writer.
• fwob: the primary library facade with auto-detecting Reader, Writer, Editor, Maintenance, and Organizer APIs, plus the command-line tool.

The logical Rust API is documented in docs/api.md.

Installation

Install the command-line tool from crates.io:

cargo install fwob

Library crates are available separately as fwob, fwob-core, fwob-derive, fwob-v1, and fwob-v2.

Library Quick Start

use fwob::{Reader, Writer};
use fwob_core::{Field, FieldType, Schema};

let schema = Schema::new(
    "Tick",
    vec![Field::new("time", FieldType::SignedInteger, 8, 0)],
    0,
)?;

let mut writer = Writer::create_v2(
    "ticks.fwob",
    schema,
    fwob_v2::WriterOptions::new("prices"),
)?;
writer.append_frame(&123_i64.to_le_bytes())?;
writer.finish()?;

let mut reader = Reader::open("ticks.fwob")?;
let first = reader.first_frame()?.expect("one frame");
assert_eq!(first.bytes(), 123_i64.to_le_bytes());
# Ok::<(), Box<dyn std::error::Error>>(())

See docs/api.md for reading, appending, editing, maintenance, organization, typed-frame, and format-specific examples.

Command Examples

fwob verify ticks.fwob
fwob inspect ticks.fwob
fwob convert ticks.fwob ticks-v2.fwob smallest 1MiB --zstd-level 9
fwob convert ticks.fwob ticks-columnar.fwob columnar-basic zstd
fwob convert v2 ticks.fwob ticks-delta.fwob columnar-delta zstd verify
fwob append ticks-v2.fwob new-ticks.fwob verify
fwob split ticks.fwob parts 1000 2000 3000
fwob concat ticks-joined.fwob parts/ticks.part0.fwob parts/ticks.part1.fwob
fwob edit ticks-joined.fwob --title Renamed --append-string NASDAQ
fwob verify ticks-v2.fwob
fwob bench range ticks-v2.fwob --first-key-i32 100 --last-key-i32 200

Positional tokens are case-sensitive. For example, v2, zstd, and 1MiB are tokens; V2, ZSTD, and 1MIB are treated as paths or values rather than their lowercase token forms.

Tuning Parameters

Status

This repository is intentionally separate from the original C# implementation: https://github.com/stcmz/Mozo.Fwob. The v1 crate is designed as a production compatibility layer rather than a one-off converter so conversions can be verified byte-for-byte against existing files.