unifier

Filesystem postbox for inter-process communication. Programs share state by writing values to files in a Unix tree, treating the filesystem as a global data structure — the same strategy used by filesystem-git-issues (directories are namespaces, files are keys, UUID-named files are messages).

No central server is required.

Sister project: Jan

Unifier is the state plane of a two-daemon multi-agent setup; jan-cli is the execution plane (cron schedules, process spawn). The two share no code — unifier has no dependency on Jan and no knowledge of Jan concepts — and are coupled only by three wire artifacts under .daemon/: notices Unifier broadcasts on events.sock, tick lifecycle calls Jan makes on tick.sock, and ordinary CLI calls agents make on unifier.sock. Unifier runs perfectly well with no scheduler attached.

The normative schema, delivery semantics (notices are best-effort hints; the board is the durable truth), and invariants live in jan-cli/docs/maintainers/ipc-contract.md. Changing Notice in src/daemon/notify.rs or the tick verbs in src/daemon/protocol.rs is a cross-project change.

State layout

Default root: ~/.local/unifier (override with $UNIFIER_HOME or --home).

~/.local/unifier/
  keys/<path>                                    # persistent key-value (global)
  mailbox/<recipient>/<uuid>.txt                 # point-to-point messages
  cron/<min>_<hour>_<dom>_<mon>_<dow>/<uuid>.txt # scheduled drops
  sqlite/<name>.sqlite                           # managed SQLite databases
  sqlite/triples.sqlite                          # reserved subject/predicate/object store
  events/<uuid>.json                             # JSON events (payload + created_at + expires_at)
  logs/<uuid>.json                               # span-tree log entries
  .daemon/
    unifier.sock                             # control plane (put/get/mail/…)
    tick.sock                                # tick lifecycle only (Jan → Unifier)
    events.sock                              # outbound notices (Unifier → Jan / watchers)
    www/<name>                               # temp files served over HTTP
    http.port                                # bound localhost web port
  chroots/
    <name>/
      keys/
      mailbox/
      cron/
      sqlite/                                    # isolated subtree (--chroot)

Cron schedule directories use five underscore-separated fields. Use * for “any” (e.g. 0_9_*_*_* = 09:00 daily).

CLI reference

Global flags:

• --home <path> / $UNIFIER_HOME — top-level store directory
• --chroot <name> / $UNIFIER_CHROOT — scope data commands to chroots/<name>/
• --no-daemon — force direct filesystem access even when a hot daemon is running
• --namespace <name> / $UNIFIER_NAMESPACE — prefix keys for this invocation (overrides a bound namespace; empty disables it)
• $UNIFIER_HTTP_PORT — preferred localhost port for the daemon web server (default 17355)
• $UNIFIER_EVENT_TTL_SECS — default event lifetime in seconds (default 86400 / 24h; 0 = never expire)

Examples

# Build
cargo build --release

# Shared key-value
unifier put app/theme dark
unifier get app/theme

# Mailbox messaging
unifier send --from builder worker "rebuild docs"
unifier poll worker --ack

# Agent envelopes + wakeup (Jan listens on events.sock)
unifier message --from myagent youragent '{"hello":"world"}'
unifier event '{"name":"prices.updated"}'          # expires in 24h
unifier event '{"name":"keep"}' --ttl 0            # never expires
unifier event '{"name":"job","ttl":60}'            # payload ttl (seconds)
unifier daemon watch   # prints {"kind":"mailbox","id":"...","from":"myagent","to":"youragent"}

# Cron drops
unifier cron 0_0_*_*_* "nightly backup"
unifier poll-cron --ack

# ACID ticks (daemon required; CLI uses .daemon/tick.sock)
unifier tick start swarm
unifier tick phase sense
unifier put board/x 1          # staged until tick end
unifier tick phase decide
unifier tick end
unifier daemon watch           # also sees {"kind":"tick","tick":1,"phase":"sense",...}

# Chroots
unifier chroot init work
unifier --chroot work put deploy/target staging
unifier --chroot work get deploy/target   # works
unifier get deploy/target                 # not found (global scope)

export UNIFIER_CHROOT=work
unifier send worker "task"

# Process-scoped key prefix (until the script exits or namespace set is called again)
unifier namespace set myscript
unifier put app/theme dark          # keys/myscript/app/theme
unifier get app/theme               # dark
unifier get /myscript/app/theme     # same key, absolute (no extra prefix)
unifier --namespace other get k     # keys/other/k for this call only
unifier namespace clear

# Managed SQLite
unifier sql create app
unifier sql exec app "CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)"
unifier sql exec app "INSERT INTO items(name) VALUES ('alpha')"
unifier sql tables app --schema
unifier sql exec app "SELECT * FROM items"
unifier sql list

# Triples (auto-creates sqlite/triples.sqlite)
unifier triple add alice knows bob
unifier triple query --subject alice
unifier sql exec triples "SELECT * FROM triples"

# Span-tree logging
SPAN=$(unifier log start pipeline --field env=prod)
STEP=$(unifier log start fetch-prices --parent $SPAN)
unifier log event $STEP "fetched 42 rows" --field count=42
unifier log end $STEP
unifier log end $SPAN
unifier log tree          # indented tree with events and fields
unifier log list          # UUID  name  open|ended

# Daemon web server (temp HTML reports)
jan report | unifier serve nightly --wrap --title "Nightly"
# => http://127.0.0.1:17355/nightly
unifier web list
unifier web url nightly
curl "$(unifier web url nightly)"
unifier web rm nightly

Installation

cargo install unifier-cli

The binary is unifier. crates.io uses unifier-cli because unifier is already taken.

Releasing

Same flow as jan-cli. From unifier/:

./scripts/release.sh              # patch bump
./scripts/release.sh --minor
./scripts/release.sh --major
./scripts/release.sh --set 1.2.3
./scripts/release.sh --dry-run
./scripts/release.sh --resume

Monorepo aliases:

mdo release
mdo release-dry-run
mdo release-minor
mdo release-major
mdo publish

This bumps Cargo.toml + CHANGELOG.md, commits those version files, runs tests, then cargo publish as unifier-cli.

Development

cargo test
cargo clippy -- -D warnings
cargo fmt

Summary of changes

Initial implementation

Created the unifier Rust CLI project with:

• Filesystem-as-store model — inspired by filesystem-git-issues: entities map to directories, scalar values to files, messages to UUID .txt drops.
• State root at ~/.local/unifier with $UNIFIER_HOME / --home override.
• Key-value store — put, get, del under keys/, with atomic writes for put.
• Mailbox — send, poll, ack for point-to-point messages under mailbox/<recipient>/.
• Cron postbox — cron, poll-cron, list using five-field schedule directory names matched against the current clock.
• Path safety — keys and recipients reject ..; message ack resolves paths relative to the store root.
• Tests — unit tests for cron parsing/matching; integration tests for put/get, mailbox, cron, and root.
• project.meta.yaml — monorepo metadata, build scripts, and roadmap for future applications (zshrc hooks, agent pipelines, job queues, etc.).

Chroot support

Added isolated subtrees so multiple programs can share one global store without seeing each other's data:

• On-disk layout — chroots/<name>/ contains a full copy of the layout (keys/, mailbox/, cron/).
• --chroot / $UNIFIER_CHROOT — all data commands operate only inside the named chroot; UnifierHome::path() returns the effective subtree root.
• Admin commands — chroot init <name> creates a new sandbox; chroot list enumerates existing chroots (always uses the global root).
• Scope module (scope.rs) — centralizes path validation: resolve_under_root() rejects .. and absolute escapes; used by list and ack.
• Isolation tests — verify keys in one chroot are invisible to the global store and other chroots; path escape attempts fail.
• Backward compatibility — omitting --chroot preserves the original global-root behavior.

Source layout

src/
  main.rs          # binary entry point
  lib.rs           # library + run()
  home.rs          # UnifierHome (global + effective root)
  scope.rs         # chroot path confinement
  chroot.rs        # chroot init/list
  paths.rs         # path builders for keys, mailbox, cron
  postbox.rs       # put/get/send/poll/ack operations
  cron.rs          # schedule parsing and matching
  fs_text.rs       # read/write helpers
  constants.rs     # directory name constants
  error.rs         # Error type
  cli/
    defs.rs        # clap command definitions
    mod.rs         # dispatch
tests/
  cli_integration.rs

Hot daemon

Added an in-memory hot store with on-demand disk flush:

• HotStore (store.rs) — loads keys/mailbox/cron from disk; tracks dirty entries; flush() writes only changed data
• Daemon process — unifier daemon start|run|stop|status|flush|watch; control socket at <store>/.daemon/unifier.sock; event output at <store>/.daemon/events.sock
• Mailbox envelopes — messages store {id, from, to, payload}; send/message/event broadcast a wakeup notice (to + id) so Jan can fetch mailbox/<to>/<id>.txt
• Auto-routing — data commands use the daemon when running; --no-daemon forces direct filesystem access
• Idle auto-flush — after 5s with no daemon requests, dirty state is written to disk if 1-minute loadavg ≤ 0.2 (UNIFIER_IDLE_FLUSH_SECS, UNIFIER_IDLE_MAX_LOAD; set flush secs to 0 to disable). Skips an active tick.
• Idle auto-stop — after 10 minutes with no control/event activity, no active tick, and no event subscribers, the daemon flushes and exits (UNIFIER_IDLE_STOP_SECS; set to 0 to disable). Prevents Gradle-style auto-started daemons from living forever when unused.
• Missing-home exit — if the store root directory disappears (e.g. a test TempDir is dropped), the daemon exits immediately so orphans do not pile up under /tmp.
• unifier daemon gc — kill orphan unifier daemon run processes whose --home path no longer exists (--dry-run to list only).

Event expiry

Posted events are stored as {payload, created_at, expires_at} under events/<uuid>.json so they do not accumulate forever:

• Default TTL is 24 hours ($UNIFIER_EVENT_TTL_SECS to change the default; 0 = never)
• unifier event '...' --ttl <secs> overrides (--ttl 0 never expires)
• Payload may set "ttl": 60 (seconds) or "expires_at": "<rfc3339>" (null = never)
• Expired files are deleted on daemon load, flush, and a 30s sweep

Process namespaces

namespace set writes a binding for the calling process under .daemon/namespaces/<pid>. Later key operations from that process or a descendant prefix put/get/del and tick lock names with <namespace>/. Bindings are dropped when the process exits (reaped) or namespace clear / a new namespace set runs. Keys starting with / skip the prefix. $UNIFIER_OWNER_PID overrides which process owns the binding (the CLI parent, by default).

Span-tree logging

Added unifier log for tracing Jan cron scripts and agent pipelines:

• log start <name> [--parent <id>] [-f k=v] — create a new span under logs/<uuid>.json; prints UUID for capture
• log end <id> — record ended_at on a span
• log event <span> <msg> [-f k=v] — append a timestamped event to a span's events array
• log field <span> k=v... — set or update key/value fields on a span
• log tree [--span <id>] — render the full span tree (or subtree) as indented text with events and fields
• log list — list all spans with UUID, name, and open/ended status
• Spans load directly from disk (no daemon round-trip); sorted by started_at for stable tree output

Daemon web server

Temp static files for Jan report export and quick HTML previews, plus live key reads for deep links from those pages:

• serve [name] — read stdin (or --file), write under .daemon/www/, print http://127.0.0.1:<port>/<name>
• --wrap — embed body in Unifier HTML chrome (brand header + “all reports” link)
• --ttl <secs> — auto-expire published files; GC on HTTP access / periodic sweep
• web list|url|rm|status — manage and inspect published files
• web key-url <key> — print http://127.0.0.1:<port>/keys/<key>
• GET /keys/<key> — return the live key value (application/json or text/plain); browsers get an HTML wrap when Accept prefers text/html
• GET /keys/ / GET /keys/<prefix>/ — list present keys (plain lines or HTML)
• Daemon binds localhost (preferred port 17355, or $UNIFIER_HTTP_PORT, else ephemeral); port stored in .daemon/http.port
• HTTP activity refreshes the daemon idle timer so open reports are not cut off mid-view

Daemon durability

Three fixes that agents launched by a scheduler depend on:

• One connection, many requests — the daemon serves a connection until the peer closes it (or goes quiet for 2s). It used to close after the first response, which broke poll --ack: the ack hit a broken pipe and the messages stayed in the mailbox, so the next wake reprocessed them.
• Own session — a spawned daemon calls setsid, so it survives the shell, cron job, or agent that auto-started it rather than taking a SIGHUP with it.
• Flush on signal — SIGTERM / SIGINT / SIGHUP drain hot state to disk before exit. Previously anything not yet flushed was lost.

Multi-agent example

The job pipeline lives in the personal dotfiles jan tree (dotfiles/jan/jobs.yaml, sibling of this Incubator checkout): four agents that only ever talk through Unifier.

job-scout  --(message)-->  job-analyst  --(message)-->  job-responder  --(event)-->  job-digest

job-scout runs on cron and mails new job postings; job-analyst and job-responder have no schedule at all and are woken by Jan when mail arrives for their leaf name; job-digest renders the result with serve. Each agent drains its whole mailbox per wake, so coalesced or duplicated wakeups are harmless. Manual poke: jan jobs interact run --to job-analyst --message '…'.

Managed SQLite

Databases live under sqlite/<name>.sqlite (scoped by --chroot when set). sql list / sql tables / sql exec manage and query them directly on disk (no hot-daemon round-trip). The reserved triples database stores unique subject/predicate/object rows; use triple add / triple query or sql exec triples '...'.

Source layout (updated)

src/
  store.rs         # HotStore (in-memory + flush)
  daemon/          # protocol, client, server, lifecycle
  ...

Roadmap

See project.meta.yaml for the full list. Next major themes:

• Tick-based agents (sense/decide/act lockstep, concurrency modes, differential sensing)
• Factorio factory model: isolated arms coordinated toward a shared system goal