owpurge 0.2.0

Askar wallet pruning for multi-tenant SSI platforms: a single static binary that prunes stale exchange records straight from the wallet store
owpurge-0.2.0 is not a library.

owpurge

Askar wallet pruning for multi-tenant SSI platforms — a single static binary with no runtime dependencies.

It connects straight to the Askar/PostgreSQL wallet store and deletes stale exchange records without touching the running agent controller.

Why this needs no agent framework

The wallet is an Askar store, and this binary links aries-askar directly. Credo's AskarStorageService is a thin pass-through over the same library — deleteById is a session.remove({ category, name }), and findByQuery is a Scan({ category, tagFilter, profile }).fetchAll() — so talking to Askar directly loses nothing on the delete path.

What remains of the framework dependency is a set of constants: category strings, state values, and the tenant-to-profile naming rule. Those live in src/credo.rs, each recorded with the upstream package and version it was read from.

The reason this is safe: the tool only ever deletes. It never writes a record, so the serialization and tag-transform logic — the only part of the storage layer with real behaviour — is never reached.

Build

cargo build --release          # -> target/release/owpurge  (~3.4 MB)

A native build links the host's system libraries (no OpenSSL — TLS is rustls, compiled in). The release binaries are fully static (musl): no libc dependency at all, so they run on any Linux of the right architecture — checked on CentOS 7 (glibc 2.17), Ubuntu 18.04 and Debian 10, including a Postgres connection by hostname. Copy the one file to the host and run it.

Building for the Linux EC2 host from macOS — either build on the host itself (cargo + a git checkout) or cross-compile.

Mind the architecture — check the target host rather than assuming. Graviton instances are aarch64; Intel/AMD ones are x86_64. uname -m on the host settles it.

cargo install cross
cross build --release --target aarch64-unknown-linux-musl   # Graviton / ARM hosts
cross build --release --target x86_64-unknown-linux-musl    # Intel/AMD hosts

Note Cargo.toml pins Askar to an exact git tag, so the first build needs network access to GitHub. Currently v0.4.6. Both 0.4.6 and 0.5.0 were verified to produce byte-identical purge results on a real wallet — same surviving rows, same tags — and the API surface used here is unchanged between them, so either is fine. 0.4.6 is also on crates.io, so at that version the git dependency can be replaced with a plain version = "0.4.6" if you would rather not reach GitHub at build time.

CI builds — .github/workflows/release.yml cross-compiles both Linux targets and attaches them to a GitHub Release. Pushing a v* tag builds both architectures; running the workflow by hand lets you pick one, and publishes only if you name a tag (otherwise the binaries are left as a workflow artifact). It builds the *-linux-musl targets with cross, and the build fails if the result has any dynamic dependency, so a binary that would depend on the host's glibc can never be published.

cross 0.2.5 does not run on Apple Silicon. To build a static aarch64 binary there, build natively inside Alpine (itself musl):

docker run --rm -v "$PWD":/src -w /src -e CARGO_TARGET_DIR=/src/target/alpine \
  rust:1-alpine sh -c 'apk add -q musl-dev && cargo build --release --locked'

The repo is public, so the assets download without auth — a plain curl -L of the asset URL works. gh is just the convenient route from a checkout; otherwise pass -R <owner>/<repo>:

gh release download v0.1.1 -p 'owpurge-linux-aarch64*'
sha256sum -c owpurge-linux-aarch64.sha256
chmod +x owpurge-linux-aarch64

Assets carry the owpurge-* name from v0.1.1 onward; v0.1.0 predates the rename and still has credo-purge-* assets.

Commands

Command What it does
owpurge census Read-only per-category record counts
owpurge purge Full drain: all categories, all targets, with checkpoint/resume
owpurge bulk-purge Fast parent-only purge, no child cascade
owpurge orphan-sweep DidComm messages whose parent exchange is gone
owpurge credentials Terminal credential exchanges + their message children
owpurge proofs Terminal proof exchanges + their message children
owpurge oob Out-of-band invitations
owpurge basic-messages Basic messages
owpurge question-answer Question-answer records
owpurge tenants List tenant ids
owpurge batch Run one command for several agents on this host, one after another
owpurge env-apply Update an existing agent .env from a values file (does not touch the wallet)

Global flag: --env-file <PATH> loads an agent's config from that file instead of .env.

Every setting is an environment variable with a fail-safe default; see .env.example.

ConnectionRecord is never deleted and there is deliberately no subcommand for it.

env-apply: updating an agent .env

Brings an existing dotenv file in line with a values file. Keys already present are rewritten in place (every occurrence), missing keys are appended, and every other line is left as it was. The original is copied to <file>.bak-<timestamp> first and the file keeps its permissions.

When a value from the values file would replace a different, non-empty value already in the file, both are shown and the runner is asked (Update it? [Y/n]; Enter updates, n keeps the current value). Filling a missing or empty key does not ask. Prompted keys show their current value in the prompt instead, so typing a new one is the decision. --yes updates without asking (both values are still printed), for runs with no terminal. Keys whose name looks secret are shown masked (******** plus the last four characters).

The values file uses dotenv syntax, plus these directives:

API_KEY=@random-hex:32        # 32 random bytes as hex, ONLY if API_KEY is missing/empty
PUBLIC_URL=@prompt-url        # asked on the terminal; must be http(s); trailing / stripped
SOME_SETTING=@prompt          # asked on the terminal; blank keeps the current value
LOG_FORMAT=json               # literal (write @@ for a value that starts with @)

owpurge env-apply --values values.env --env /path/to/agent/.env --dry-run
owpurge env-apply --values values.env --env /path/to/agent/.env
# --env omitted: asks for the path on the terminal
owpurge env-apply --values values.env
# values on stdin; prompts still go to the terminal:
decode-values | owpurge env-apply --values - --env /path/to/agent/.env
# non-interactive (no terminal): answer prompts up front
owpurge env-apply --values values.env --env .env --set PUBLIC_URL=https://host.example

--review-json [PATH] prints a JSON file — typically the agent's own config — before any prompt, so the person running it can check the box first. Without a path it asks for one. Any field whose name contains key, password, secret, token, seed, private or mnemonic is masked as ******** (empty values stay visible, so "unset" is distinguishable from "set"); everything else prints as-is, in file order.

@random-hex never replaces an existing value. Rotating a key that clients already hold would lock them out, so rotation stays a deliberate manual edit. The agent has to be restarted to pick up the changes.

Multi-tenant mode

PURGE_MODE=multi-tenant (the default) drains each tenant inside its own Askar profile, tenant-<id>. Target selection: TENANT_ALLOWLIST=id1,id2 beats TENANT_ID=<id>, and with neither set purge enumerates every TenantRecord in the root profile (owpurge tenants lists them). Progress is checkpointed per tenant, so an interrupted drain resumes; an unknown tenant is skipped and audited, and one tenant failing does not abort the rest. PURGE_MODE=dedicated drains the root profile of an agent deployed without multi-tenancy.

Run census on a tenant before the first live drain of a wallet, and check the two per-wallet traps found on the dedicated agent (KEY_DERIVATION_METHOD not being raw, DB_USER determining the schema): they fail silently rather than loudly.

MULTI_TENANT_ENABLED in src/config.rs is a compile-time constant, not an env var, so turning a mode off is a decision visible in a diff rather than a stray variable in a cron environment.

Several agents on one host

Each agent is its own wallet with its own env file. --env-file <PATH> points any command at one agent's file instead of .env; the file must exist, and a variable already set in the shell wins over it.

owpurge --env-file /etc/agents/issuer.env purge

batch runs one command for many agents, one after another:

owpurge batch --dir /etc/agents                       # every *.env in the directory, purge
owpurge batch --env a.env --env b.env --run census    # explicit files, a different command
  • Each agent runs as a separate process with a cleared environment, configured only by its own file. Its DRY_RUN, TTL_DAYS and tenant settings apply to it alone, so a batch can mix a dry-run agent and a live one without either leaking into the other.
  • Everything is validated before the first agent runs: unreadable files, a missing WALLET_ID, or two files naming the same wallet on the same host are refused.
  • A failing agent does not stop the rest (--stop-on-error changes that). The summary lists each agent, and the exit status is non-zero if any failed.
  • Checkpoints are keyed by WALLET_ID (purge-checkpoint-<wallet>.json), so agents never share one. All agents append to the same audit CSV by default, each row carrying its wallet_id; set AUDIT_CSV per agent to split them.
  • --run takes any wallet command except env-apply.

Safety gates

  • DRY_RUN defaults to dry-run. Only the literal DRY_RUN=false deletes.
  • bulk-purge additionally requires BULK_CONFIRM=true, checked before the store is even opened.
  • Stale-incomplete purge needs PURGE_STALE_INCOMPLETE=true, applies to proofs only, and uses a separate, more conservative TTL. A 0-day floor needs a second key (STALE_ALLOW_ZERO_TTL=true).

CSV audit trail

Every purge appends to AUDIT_CSV (default ./logs/purge-audit.csv), one row per target, flushed as that target finishes — not buffered to the end. A 29-tenant drain that dies on tenant 20 leaves 19 durable rows plus a failed row for the one that broke. Parent directories are created automatically. --no-audit turns it off.

started_at,finished_at,duration_s,command,status,mode,purge_mode,target,wallet_id,
ttl_days,stale_incomplete_ttl_days,purge_stale_incomplete,throttle_ms,heartbeat_every,
bulk_purge_batch_size,orphan_scan_batch_size,orphan_delete_batch_size,db_max_connections,
server_side_delete,
credentials_parents,credentials_children,proofs_parents,proofs_children,oob_records,
orphans_deleted,basic_messages,question_answers,total_records,error

2026-09-29T22:10:04+00:00,2026-09-29T22:11:31+00:00,87,purge,ok,live,dedicated,root,<wallet-id>,30,,false,250,20,500,2000,500,50,false,412,1236,88,264,17,3390,0,2,5409,
2026-09-29T22:11:31+00:00,2026-09-29T22:11:33+00:00,2,purge,failed,live,dedicated,root,<wallet-id>,30,,false,250,20,500,2000,500,50,false,0,0,0,0,0,0,0,0,0,"db timeout, retrying"

Notes on reading it:

  • Parameters are denormalised onto every row on purpose. A row has to be interpretable on its own months later, without cross-referencing shell history.
  • status is ok, failed, or skipped (target named but no Askar profile exists). Failures get a row rather than a gap — a gap is indistinguishable from "never attempted".
  • total_records counts parents and cascaded children, so it should track the actual drop in wallet row count.
  • mode is dry-run or live. Dry-run rows record what would have gone.
  • A multi-tenant run writes no grand-total row — the per-tenant rows sum to it, and a total row would double-count any SUM() over the column.
  • Rows are audited before the checkpoint is written. If the process dies between the two, the tenant is re-purged next run (deletes are already-gone tolerant) and the CSV shows both attempts. The reverse order could checkpoint a tenant with no record it ran.
  • census writes nothing — the file is a record of deletions.

Quick queries:

# What did we delete last night, by tenant?
awk -F, '$1 ~ /^2026-09-28/ && $5=="ok" {print $8, $28}' logs/purge-audit.csv

# Anything fail this month?
grep ',failed,' logs/purge-audit.csv

Design notes

  • Scans stream. Askar's Scan is a Stream and each page drops when it leaves scope, so memory stays flat across a whole scan regardless of wallet size. There is no per-batch buffer to free and no heap ceiling to raise.
  • A delete batch is one transaction — N statements, one commit, one fsync — rather than N autocommitting deletes competing for DB_MAX_CONNECTIONS.
  • The cascade child lookup is one query per batch. associatedRecordId is a real Askar tag, so a batch of 100 parents resolves in a single any_of filter instead of 100 round trips.
  • The orphan sweep deserializes JSON only for orphan candidates. It reads the parent link off entry.tags and pays for serde_json only when a record actually looks orphaned, which is a small fraction of what it scans.
  • Scans pass OrderBy::Id. Paging across separate scan calls is only stable with an explicit sort; without one, a resumed scan can silently return short counts rather than erroring.

Opt-in: --server-side-delete. Collapses a track into a single server-side DELETE ... WHERE. It only applies where the whole (category, state) set is eligible, i.e. TTL_DAYS=0, and never to the non-reusable OOB track. An age filter reads updatedAt, which lives inside the encrypted record value rather than in a tag, so it cannot be pushed into SQL. With any TTL_DAYS > 0 the flag is ignored.

Category drift, and the guard against it

Category and state strings are hardcoded in src/credo.rs rather than imported from the agent framework. That buys the zero-dependency binary, but it means an upstream rename cannot surface as a compile error — it would surface as a category that silently matches nothing.

store::preflight is the guard: if a wallet holds records under the protected categories (ConnectionRecord, DidRecord — never purged) but every purgeable category reads zero, that is far more likely to be category drift than a genuinely empty target, and it says so loudly rather than reporting a clean run.

After any framework upgrade on the agent, re-check the .type statics against src/credo.rs, which records the upstream package and version each one came from.

Host notes: the dedicated agent

PURGE_MODE=dedicated targets a single-profile Askar store. On this platform that store is not RDS — it is a postgres:16 container running on the agent host, with its data on a bind mount. Confirm the specifics for your deployment; they differ from the multi-tenant defaults in .env.example:

multi-tenant (RDS) dedicated (containerised)
DB_HOST the RDS endpoint the agent host, or localhost on-box
DB_PORT 5432 often non-default — check the published port
DB_USER postgres the user the agent connects as — see below
PURGE_MODE multi-tenant dedicated

Three things to establish per wallet before a first run, each of which fails in a way that does not look like a configuration problem:

1. The key derivation method is often not raw. Credo defaults to KdfMethod.Argon2IMod whenever the agent never sets keyDerivationMethod, and Askar refuses to open a store with the wrong method rather than degrading. Read the real value out of the store — it is not secret:

SELECT split_part(value, '?', 1) FROM config WHERE name = 'key';
-- 'kdf:argon2i:13:mod' -> KEY_DERIVATION_METHOD=kdf:argon2i:mod
-- 'raw'                -> KEY_DERIVATION_METHOD=raw

2. Set DB_SCHEMA — otherwise DB_USER silently decides where the store is. Askar creates its tables in schema.unwrap_or(username) at provision time and locates them again through CURRENT_SCHEMAS(false) — the Postgres search_path, default "$user", public. Credo never sets the schema URI parameter, so a Credo-provisioned store lives in a schema named after whichever user the agent connects as, and connecting as anyone else reports no store in a perfectly good database:

DB_USER=postgres, no DB_SCHEMA   ->  relation "config" does not exist
DB_USER=postgres, DB_SCHEMA set  ->  store opens normally

Askar applies DB_SCHEMA as search_path on every pooled connection, so setting it pins where the store is and reduces DB_USER to authentication. Find the value with:

SELECT table_schema FROM information_schema.tables
WHERE table_name IN ('config','profiles','items','items_tags')
GROUP BY table_schema HAVING count(*) >= 3;

This does diverge from how the agent opens the store, so the value must be right — read it per wallet rather than assuming.

3. Disk is often shared. A containerised wallet typically sits on the host's root volume alongside every other service on that box. A large purge's WAL growth and post-vacuum bloat land on that shared volume, so watch free space rather than assuming RDS-style isolation.

Also confirm which Credo framework version the agent image corresponds to before trusting the hardcoded category names in src/credo.rs — they were read from Credo 0.7. store::preflight catches a mismatch, but checking first is cheaper.

See testing/ for cloning a wallet locally to test against.

Rolling it out

Validation is a counting exercise — establish the numbers, then delete.

  1. owpurge census against a target. Record every category total and per-state count as the baseline.
  2. DRY_RUN=true owpurge bulk-purge, then DRY_RUN=true owpurge purge on one small target. Eligible counts per track should be consistent with the census.
  3. Go live on that one small target, with a census before and after. The protected category counts must be unchanged, and total_records in the CSV should equal the drop in the purgeable totals.
  4. Work up to the largest target last.
cargo test        # 30 tests: category/state constant invariants, fail-safe TTL
                  # resolution, store-URI encoding, CSV escaping and appending,
                  # that both purge modes are allowed, per-wallet checkpoint paths,
                  # and env-apply's parsing, in-place editing, key preservation
                  # review masking and update confirmation

Validated against a production-scale wallet

The full drain sequence (terminal bulk-purge → orphan sweep → stale-incomplete → orphan sweep → basic/Q&A, at TTL_DAYS=20) was run against a local clone of a real single-profile wallet holding 71,379 items.

The result was checked by fingerprinting the database rather than by trusting the reported counts — md5 over all surviving record ids, over the category distribution, and over the full items_tags contents. The protected categories came through untouched: ConnectionRecord (2,802) and DidRecord (5,612), exactly as before the run. The whole sequence took 29s, about 11s of which was THROTTLE_MS sleeps.

See testing/ to reproduce.