# owpurge
Askar wallet pruning for multi-tenant SSI platforms — a single static binary with no
runtime dependencies.
It connects straight to the Askar/PostgreSQL wallet store and deletes stale exchange
records without touching the running agent controller.
---
## Why this needs no agent framework
The wallet is an [Askar](https://github.com/openwallet-foundation/askar) store, and this
binary links `aries-askar` directly. Credo's `AskarStorageService` is a thin pass-through
over the same library — `deleteById` is a `session.remove({ category, name })`, and
`findByQuery` is a `Scan({ category, tagFilter, profile }).fetchAll()` — so talking to
Askar directly loses nothing on the delete path.
What remains of the framework dependency is a set of constants: category strings, state
values, and the tenant-to-profile naming rule. Those live in
[`src/credo.rs`](src/credo.rs), each recorded with the upstream package and version it
was read from.
**The reason this is safe: the tool only ever deletes.** It never writes a record, so the
serialization and tag-transform logic — the only part of the storage layer with real
behaviour — is never reached.
---
## Build
```sh
cargo build --release # -> target/release/owpurge (~3.4 MB)
```
A native build links the host's system libraries (no OpenSSL — TLS is rustls,
compiled in). The **release binaries are fully static** (musl): no libc dependency at
all, so they run on any Linux of the right architecture — checked on CentOS 7
(glibc 2.17), Ubuntu 18.04 and Debian 10, including a Postgres connection by
hostname. Copy the one file to the host and run it.
**Building for the Linux EC2 host from macOS** — either build on the host itself
(`cargo` + a git checkout) or cross-compile.
Mind the architecture — check the target host rather than assuming. Graviton
instances are `aarch64`; Intel/AMD ones are `x86_64`. `uname -m` on the host settles
it.
```sh
cargo install cross
cross build --release --target aarch64-unknown-linux-musl # Graviton / ARM hosts
cross build --release --target x86_64-unknown-linux-musl # Intel/AMD hosts
```
Note `Cargo.toml` pins Askar to an exact **git tag**, so the first build needs network
access to GitHub. Currently `v0.4.6`. Both `0.4.6` and `0.5.0` were verified to produce
byte-identical purge results on a real wallet — same surviving rows, same tags — and
the API surface used here is unchanged between them, so either is fine. `0.4.6` is also
on crates.io, so at that version the git dependency can be replaced with a plain
`version = "0.4.6"` if you would rather not reach GitHub at build time.
**CI builds** — `.github/workflows/release.yml` cross-compiles both Linux targets and
attaches them to a GitHub Release. Pushing a `v*` tag builds both architectures;
running the workflow by hand lets you pick one, and publishes only if you name a tag
(otherwise the binaries are left as a workflow artifact). It builds the `*-linux-musl`
targets with `cross`, and the build fails if the result has any dynamic dependency, so
a binary that would depend on the host's glibc can never be published.
`cross` 0.2.5 does not run on Apple Silicon. To build a static `aarch64` binary there,
build natively inside Alpine (itself musl):
```sh
docker run --rm -v "$PWD":/src -w /src -e CARGO_TARGET_DIR=/src/target/alpine \
rust:1-alpine sh -c 'apk add -q musl-dev && cargo build --release --locked'
```
The repo is public, so the assets download without auth — a plain `curl -L` of the
asset URL works. `gh` is just the convenient route from a checkout; otherwise pass
`-R <owner>/<repo>`:
```sh
gh release download v0.1.1 -p 'owpurge-linux-aarch64*'
sha256sum -c owpurge-linux-aarch64.sha256
chmod +x owpurge-linux-aarch64
```
Assets carry the `owpurge-*` name from v0.1.1 onward; v0.1.0 predates the rename and
still has `credo-purge-*` assets.
---
## Commands
| `owpurge census` | Read-only per-category record counts |
| `owpurge purge` | Full drain: all categories, all targets, with checkpoint/resume |
| `owpurge bulk-purge` | Fast parent-only purge, no child cascade |
| `owpurge orphan-sweep` | DidComm messages whose parent exchange is gone |
| `owpurge credentials` | Terminal credential exchanges + their message children |
| `owpurge proofs` | Terminal proof exchanges + their message children |
| `owpurge oob` | Out-of-band invitations |
| `owpurge basic-messages` | Basic messages |
| `owpurge question-answer` | Question-answer records |
| `owpurge tenants` | List tenant ids |
| `owpurge batch` | Run one command for several agents on this host, one after another |
| `owpurge env-apply` | Update an existing agent `.env` from a values file (does not touch the wallet) |
Global flag: `--env-file <PATH>` loads an agent's config from that file instead of `.env`.
Every setting is an environment variable with a fail-safe default; see
[`.env.example`](.env.example).
`ConnectionRecord` is never deleted and there is deliberately no subcommand for it.
### `env-apply`: updating an agent `.env`
Brings an existing dotenv file in line with a values file. Keys already present are
rewritten in place (every occurrence), missing keys are appended, and every other line
is left as it was. The original is copied to `<file>.bak-<timestamp>` first and the
file keeps its permissions.
When a value from the values file would replace a **different, non-empty** value already
in the file, both are shown and the runner is asked (`Update it? [Y/n]`; Enter updates,
`n` keeps the current value). Filling a missing or empty key does not ask. Prompted keys
show their current value in the prompt instead, so typing a new one is the decision.
`--yes` updates without asking (both values are still printed), for runs with no
terminal. Keys whose name looks secret are shown masked (`********` plus the last four
characters).
The values file uses dotenv syntax, plus these directives:
```sh
API_KEY=@random-hex:32 # 32 random bytes as hex, ONLY if API_KEY is missing/empty
PUBLIC_URL=@prompt-url # asked on the terminal; must be http(s); trailing / stripped
SOME_SETTING=@prompt # asked on the terminal; blank keeps the current value
LOG_FORMAT=json # literal (write @@ for a value that starts with @)
```
```sh
owpurge env-apply --values values.env --env /path/to/agent/.env --dry-run
owpurge env-apply --values values.env --env /path/to/agent/.env
# --env omitted: asks for the path on the terminal
owpurge env-apply --values values.env
# values on stdin; prompts still go to the terminal:
owpurge env-apply --values values.env --env .env --set PUBLIC_URL=https://host.example
```
`--review-json [PATH]` prints a JSON file — typically the agent's own config — before
any prompt, so the person running it can check the box first. Without a path it asks
for one. Any field whose name contains `key`, `password`, `secret`, `token`, `seed`,
`private` or `mnemonic` is masked as `********` (empty values stay visible, so "unset"
is distinguishable from "set"); everything else prints as-is, in file order.
`@random-hex` never replaces an existing value. Rotating a key that clients already hold
would lock them out, so rotation stays a deliberate manual edit. The agent has to be
restarted to pick up the changes.
### Multi-tenant mode
`PURGE_MODE=multi-tenant` (the default) drains each tenant inside its own Askar profile,
`tenant-<id>`. Target selection: `TENANT_ALLOWLIST=id1,id2` beats `TENANT_ID=<id>`, and
with neither set `purge` enumerates every `TenantRecord` in the root profile
(`owpurge tenants` lists them). Progress is checkpointed per tenant, so an interrupted
drain resumes; an unknown tenant is skipped and audited, and one tenant failing does not
abort the rest. `PURGE_MODE=dedicated` drains the root profile of an agent deployed
without multi-tenancy.
Run `census` on a tenant before the first live drain of a wallet, and check the two
per-wallet traps found on the dedicated agent (`KEY_DERIVATION_METHOD` not being `raw`,
`DB_USER` determining the schema): they fail silently rather than loudly.
`MULTI_TENANT_ENABLED` in `src/config.rs` is a compile-time constant, not an env var, so
turning a mode off is a decision visible in a diff rather than a stray variable in a cron
environment.
### Several agents on one host
Each agent is its own wallet with its own env file. `--env-file <PATH>` points any command
at one agent's file instead of `.env`; the file must exist, and a variable already set in
the shell wins over it.
```sh
owpurge --env-file /etc/agents/issuer.env purge
```
`batch` runs one command for many agents, one after another:
```sh
owpurge batch --dir /etc/agents # every *.env in the directory, purge
owpurge batch --env a.env --env b.env --run census # explicit files, a different command
```
- Each agent runs as a separate process with a cleared environment, configured only by
its own file. Its `DRY_RUN`, `TTL_DAYS` and tenant settings apply to it alone, so a
batch can mix a dry-run agent and a live one without either leaking into the other.
- Everything is validated before the first agent runs: unreadable files, a missing
`WALLET_ID`, or two files naming the same wallet on the same host are refused.
- A failing agent does not stop the rest (`--stop-on-error` changes that). The summary
lists each agent, and the exit status is non-zero if any failed.
- Checkpoints are keyed by `WALLET_ID` (`purge-checkpoint-<wallet>.json`), so agents never
share one. All agents append to the same audit CSV by default, each row carrying its
`wallet_id`; set `AUDIT_CSV` per agent to split them.
- `--run` takes any wallet command except `env-apply`.
### Safety gates
- `DRY_RUN` defaults to dry-run. **Only** the literal `DRY_RUN=false` deletes.
- `bulk-purge` additionally requires `BULK_CONFIRM=true`, checked before the store is
even opened.
- Stale-incomplete purge needs `PURGE_STALE_INCOMPLETE=true`, applies to **proofs
only**, and uses a separate, more conservative TTL. A 0-day floor needs a second key
(`STALE_ALLOW_ZERO_TTL=true`).
---
## CSV audit trail
Every purge appends to `AUDIT_CSV` (default `./logs/purge-audit.csv`), **one row per
target, flushed as that target finishes** — not buffered to the end. A 29-tenant drain
that dies on tenant 20 leaves 19 durable rows plus a `failed` row for the one that
broke. Parent directories are created automatically. `--no-audit` turns it off.
```
started_at,finished_at,duration_s,command,status,mode,purge_mode,target,wallet_id,
ttl_days,stale_incomplete_ttl_days,purge_stale_incomplete,throttle_ms,heartbeat_every,
bulk_purge_batch_size,orphan_scan_batch_size,orphan_delete_batch_size,db_max_connections,
server_side_delete,
credentials_parents,credentials_children,proofs_parents,proofs_children,oob_records,
orphans_deleted,basic_messages,question_answers,total_records,error
```
```csv
2026-09-29T22:10:04+00:00,2026-09-29T22:11:31+00:00,87,purge,ok,live,dedicated,root,<wallet-id>,30,,false,250,20,500,2000,500,50,false,412,1236,88,264,17,3390,0,2,5409,
2026-09-29T22:11:31+00:00,2026-09-29T22:11:33+00:00,2,purge,failed,live,dedicated,root,<wallet-id>,30,,false,250,20,500,2000,500,50,false,0,0,0,0,0,0,0,0,0,"db timeout, retrying"
```
Notes on reading it:
- **Parameters are denormalised onto every row on purpose.** A row has to be
interpretable on its own months later, without cross-referencing shell history.
- **`status`** is `ok`, `failed`, or `skipped` (target named but no Askar profile
exists). Failures get a row rather than a gap — a gap is indistinguishable from
"never attempted".
- **`total_records`** counts parents *and* cascaded children, so it should track the
actual drop in wallet row count.
- **`mode`** is `dry-run` or `live`. Dry-run rows record what *would* have gone.
- **A multi-tenant run writes no grand-total row** — the per-tenant rows sum to it, and
a total row would double-count any `SUM()` over the column.
- Rows are audited *before* the checkpoint is written. If the process dies between the
two, the tenant is re-purged next run (deletes are already-gone tolerant) and the CSV
shows both attempts. The reverse order could checkpoint a tenant with no record it ran.
- `census` writes nothing — the file is a record of deletions.
Quick queries:
```sh
# What did we delete last night, by tenant?
awk -F, '$1 ~ /^2026-09-28/ && $5=="ok" {print $8, $28}' logs/purge-audit.csv
# Anything fail this month?
grep ',failed,' logs/purge-audit.csv
```
---
## Design notes
- **Scans stream.** Askar's `Scan` is a `Stream` and each page drops when it leaves
scope, so memory stays flat across a whole scan regardless of wallet size. There is no
per-batch buffer to free and no heap ceiling to raise.
- **A delete batch is one transaction** — N statements, one commit, one fsync — rather
than N autocommitting deletes competing for `DB_MAX_CONNECTIONS`.
- **The cascade child lookup is one query per batch.** `associatedRecordId` is a real
Askar tag, so a batch of 100 parents resolves in a single `any_of` filter instead of
100 round trips.
- **The orphan sweep deserializes JSON only for orphan candidates.** It reads the parent
link off `entry.tags` and pays for `serde_json` only when a record actually looks
orphaned, which is a small fraction of what it scans.
- **Scans pass `OrderBy::Id`.** Paging across separate scan calls is only stable with an
explicit sort; without one, a resumed scan can silently return short counts rather
than erroring.
**Opt-in: `--server-side-delete`.** Collapses a track into a single server-side
`DELETE ... WHERE`. It only applies where the whole `(category, state)` set is eligible,
i.e. **`TTL_DAYS=0`**, and never to the non-reusable OOB track. An age filter reads
`updatedAt`, which lives inside the encrypted record value rather than in a tag, so it
cannot be pushed into SQL. With any `TTL_DAYS > 0` the flag is ignored.
---
## Category drift, and the guard against it
Category and state strings are hardcoded in [`src/credo.rs`](src/credo.rs) rather than
imported from the agent framework. That buys the zero-dependency binary, but it means an
upstream rename cannot surface as a compile error — it would surface as a category that
silently matches nothing.
`store::preflight` is the guard: if a wallet holds records under the **protected**
categories (`ConnectionRecord`, `DidRecord` — never purged) but every purgeable category
reads zero, that is far more likely to be category drift than a genuinely empty target,
and it says so loudly rather than reporting a clean run.
After any framework upgrade on the agent, re-check the `.type` statics against
`src/credo.rs`, which records the upstream package and version each one came from.
---
## Host notes: the dedicated agent
`PURGE_MODE=dedicated` targets a single-profile Askar store. On this platform that
store is **not RDS** — it is a `postgres:16` container running on the agent host, with
its data on a bind mount. Confirm the specifics for your deployment; they differ from
the multi-tenant defaults in `.env.example`:
| `DB_HOST` | the RDS endpoint | the agent host, or `localhost` on-box |
| `DB_PORT` | `5432` | often **non-default** — check the published port |
| `DB_USER` | `postgres` | the user the agent connects as — **see below** |
| `PURGE_MODE` | `multi-tenant` | `dedicated` |
Three things to establish per wallet before a first run, each of which fails in a way
that does not look like a configuration problem:
**1. The key derivation method is often not `raw`.** Credo defaults to
`KdfMethod.Argon2IMod` whenever the agent never sets `keyDerivationMethod`, and Askar
refuses to open a store with the wrong method rather than degrading. Read the real
value out of the store — it is not secret:
```sql
SELECT split_part(value, '?', 1) FROM config WHERE name = 'key';
-- 'kdf:argon2i:13:mod' -> KEY_DERIVATION_METHOD=kdf:argon2i:mod
-- 'raw' -> KEY_DERIVATION_METHOD=raw
```
**2. Set `DB_SCHEMA` — otherwise `DB_USER` silently decides where the store is.**
Askar creates its tables in `schema.unwrap_or(username)` at provision time and locates
them again through `CURRENT_SCHEMAS(false)` — the Postgres `search_path`, default
`"$user", public`. Credo never sets the `schema` URI parameter, so a Credo-provisioned
store lives in a schema named after whichever user the agent connects as, and
connecting as anyone else reports **no store** in a perfectly good database:
```
DB_USER=postgres, no DB_SCHEMA -> relation "config" does not exist
DB_USER=postgres, DB_SCHEMA set -> store opens normally
```
Askar applies `DB_SCHEMA` as `search_path` on every pooled connection, so setting it
pins where the store is and reduces `DB_USER` to authentication. Find the value with:
```sql
SELECT table_schema FROM information_schema.tables
WHERE table_name IN ('config','profiles','items','items_tags')
GROUP BY table_schema HAVING count(*) >= 3;
```
This does diverge from how the agent opens the store, so the value must be right —
read it per wallet rather than assuming.
**3. Disk is often shared.** A containerised wallet typically sits on the host's root
volume alongside every other service on that box. A large purge's WAL growth and
post-vacuum bloat land on that shared volume, so watch free space rather than assuming
RDS-style isolation.
Also confirm which **Credo framework** version the agent image corresponds to before
trusting the hardcoded category names in `src/credo.rs` — they were read from Credo
0.7. `store::preflight` catches a mismatch, but checking first is cheaper.
See [`testing/`](testing/) for cloning a wallet locally to test against.
## Rolling it out
Validation is a counting exercise — establish the numbers, then delete.
1. `owpurge census` against a target. Record every category total and per-state
count as the baseline.
2. `DRY_RUN=true owpurge bulk-purge`, then `DRY_RUN=true owpurge purge` on one
small target. Eligible counts per track should be consistent with the census.
3. Go live on that one small target, with a census before and after. The **protected**
category counts must be unchanged, and `total_records` in the CSV should equal the
drop in the purgeable totals.
4. Work up to the largest target last.
```sh
cargo test # 30 tests: category/state constant invariants, fail-safe TTL
# resolution, store-URI encoding, CSV escaping and appending,
# that both purge modes are allowed, per-wallet checkpoint paths,
# and env-apply's parsing, in-place editing, key preservation
# review masking and update confirmation
```
## Validated against a production-scale wallet
The full drain sequence (terminal bulk-purge → orphan sweep → stale-incomplete → orphan
sweep → basic/Q&A, at `TTL_DAYS=20`) was run against a local clone of a real
single-profile wallet holding 71,379 items.
The result was checked by fingerprinting the database rather than by trusting the
reported counts — md5 over all surviving record ids, over the category distribution, and
over the full `items_tags` contents. The protected categories came through untouched:
`ConnectionRecord` (2,802) and `DidRecord` (5,612), exactly as before the run. The whole
sequence took 29s, about 11s of which was `THROTTLE_MS` sleeps.
See [`testing/`](testing/) to reproduce.