I/O Pimdir

pimdir store and operator CLI for Rust: a SQLite and content-addressed blob storage backend for io-replica

This project is composed of 3 feature-gated layers:

• Low-level I/O-free core: no_std-compatible schema, statements and model-to-column encodings, reusable by any implementation
• Mid-level std client: PimdirStore, which runs the statements against SQLite and the blob files, servicing the io-replica storage seam
• High-level CLI: the pimdir binary, the operator front-end over a store (requires the cli feature)

Table of contents

• Features
• Specification
• Installation
• Usage
• Examples
• AI policy
• License
• Social
• Contributing
• Sponsoring

Features

• Portable store: a single SQLite index plus a content-addressed blob directory, readable by any conformant pimdir implementation.
• Deduplicated bodies: each body is stored once by content hash, so a message filed in two mailboxes costs one copy.
• Offline-first: keeps the shared item and a per-source base, the raw material a sync engine reconciles against.
• Short public ids: one small, store-global id per message, shared across every collection and never reused.
• Crash-safe writes: one transaction per batch, bodies durable before the rows that reference them, and blobs garbage collected inside it.
• Retention: a removal retires an item instead of destroying it, hidden from every read and from the sync, until an explicit purge reclaims it.
• Action queue: processes that do not own the store request mutations by appending actions the owner applies exactly once, with parked failures queryable and collection generations carrying the handle-space epoch to readers.
• Operator CLI: inspect a store while a sync is running, read the trash, restore or purge an item, prune the queue, check for orphan blobs and dump the whole store (requires the cli feature).
• no_std core: the schema, statements and encodings need no allocator beyond alloc and pull SQLite in only behind the client feature.

[!TIP] io-pimdir is written in Rust and uses cargo features to gate each layer. The default feature set is declared in Cargo.toml or on docs.rs.

Specification

io-pimdir implements the pimdir on-disk store specification: a SQLite database plus a content-addressed blob directory, with a canonical schema and forward-only migrations. The spec is the cross-implementation contract, so a store written here is readable by any other conformant implementation (a native Android SQLite store, for example). The sync model it services (a shared item, a per-source base, detail levels, conflicts) lives in io-replica.

Installation

The CLI binary pimdir has not been officially released yet. Install it from crates.io with cargo:

cargo install io-pimdir --locked --features cli

To use io-pimdir as a library, add it to your Cargo.toml: the cli feature is not part of the defaults, so a library consumer never compiles the binary or its terminal dependencies.

Usage

The pimdir binary is to a store what sqlite3 is to a database: an operator and debugging tool, not an end-user client. It is kind-agnostic and never interprets item content, so it prints ids, flags, levels and the raw meta, and exports raw bytes; rendering a message or a contact belongs to himalaya and cardamum. Reads open the store read-only, so inspecting a store mid-sync is always safe. A few real-world invocations:

pimdir -s ~/mail store info
pimdir -s ~/mail collection list
pimdir -s ~/mail item list INBOX --retained
pimdir -s ~/mail item restore 42
pimdir -s ~/mail item purge --older-than 90d
pimdir -s ~/mail queue list --parked
pimdir -s ~/mail check

Run pimdir --help for the full command tree and flags, and add --json to any command for machine-readable output. The CLI contract (the roles it opens a store with, its verb surface, its confirmation rules) lives in cairn/spec/cli.md. See documentation at docs.rs.

Examples

See complete examples at ./tests.

License

This project is licensed under either of:

• MIT license
• Apache License, Version 2.0

Social

• Chat on Matrix
• News on Mastodon or RSS
• Mail at pimalaya.org@posteo.net

Sponsoring

Special thanks to the NLnet foundation and the European Commission that have been financially supporting the project for years:

• 2022 → 2023: NGI Assure
• 2023 → 2024: NGI Zero Entrust
• 2024 → 2026: NGI Zero Core
• 2026 → 2027: NGI Zero Commons Fund

This program is part of Pimalaya, free software funded entirely by grants and donations. If you find it useful, consider sponsoring its development: