pond-db 0.16.0

Lossless storage and search for sessions from any AI agent client
docs.rs failed to build pond-db-0.16.0
Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.
Visit the last successful build: pond-db-0.15.1

pond

CI crates.io docs License

"I know we discussed that before. Why can't I find that damn conversation?"

Pond makes every AI agent session you've ever run - Claude Code, Codex, any tool, any machine - searchable in one place.

Your agent history is already on your disk: thousands of sessions full of decisions, fixes, and dead ends - scattered across tools that can't search them. Pond ingests them all automatically and losslessly into storage you own (a local dir or your own S3 bucket), makes the whole corpus searchable and SQL-queryable, and hands that recall back to your agents over MCP - so "how did we fix this before?" is a query, not an archaeology dig. Sessions stop being locked to the tool that created them: any session can be restored into any supported client and continued there.

brew install tenequm/tap/pond

Or prompt your agent: "Please install and set up pond (see github.com/tenequm/pond)."

Status: pre-v1. Schemas, wire shapes, and config keys are subject to breaking change until v1. Full documentation lives at pond.locker; the contract is docs/spec.md.

Quickstart

Install, run guided setup, and ingest your local sessions:

brew install tenequm/tap/pond
pond init   # guided setup: storage, adapters, MCP + agent skill, optional schedule
pond sync   # ingest and index - every enabled adapter

pond init registers pond as an MCP server for detected clients and installs the bundled pond skill for Claude Code (by hand: claude mcp add -s user pond -- pond mcp, codex mcp add pond -- pond mcp; skill: save pond skill output to ~/.claude/skills/pond/SKILL.md). Then ask your agent - real prompts from daily use:

check in pond how we solved this before, then apply the same fix here
where we left off yesterday - check pond, then continue
are you sure that won't break X? check in pond how we struggled with exactly this

Sessions are picked up automatically from Claude Code, the Claude desktop app (local agent mode), Codex CLI, opencode, pi-coding-agent, oh-my-pi, OpenClaw, NanoClaw, Hermes Agent, letta-code, and grok-build. A Claude.ai data export imports with pond sync claude-ai-export --path <path> (manual download, so not auto-discovered).

Isn't this another memory tool?

No - it's the layer underneath one. Memory tools store what they decided you'd need - facts, summaries, filed chunks; the sessions themselves are gone. Pond keeps the sessions: every message, tool call, and result, value-complete, cross-client, in storage you own, never pruned - searchable over MCP and restorable into any client. Memory is a derived view you can always rebuild from an archive; an archive can never be rebuilt from memories.

Three kinds of tool get called "memory". Side by side:

pond Session search (ctx, deja-vu, cass) Memory layers (Mem0, Letta)
History from before install yes yes no
What is kept the whole session a search index ¹ extracted facts
Where it lives local dir or S3 bucket local index the tool's database
After the harness deletes the file still there gone at next refresh ¹ only the extract
Several machines one shared bucket pulled into one machine ² shared server or cloud
Agent access CLI, MCP, HTTP, SQL CLI, MCP ³ HTTP, SDK, MCP

¹ cass also mirrors the raw files, so they outlive the source. ² deja-vu copies records between machines over ssh; cass pulls other hosts' session files over ssh/rsync into its local index; ctx is single-machine. ³ cass has no MCP server.

Pick session search for fast local recall. Pick a memory layer when the agent should carry distilled facts, not the record. Pick pond when you want the sessions themselves, in storage you own, from every machine you run.

Full comparison, with receipts per tool: pond.locker/compare. Every cell is a claim about a specific version of someone else's project. If one has gone stale, open an issue and it gets fixed the same day.

Background

Every agentic CLI ships its own session format and its own search surface. Switching tools means losing history. Replaying a Claude Code session in another provider's tooling means re-translating the wire shape by hand. Hosted multi-tenant deployments rebuild the same storage layer from scratch.

Pond is the storage and retrieval layer that sits underneath. Every adapter is a bidirectional codec between a client format and one canonical schema, so any session can be restored by any adapter - it need not return to the client that produced it. Storage, search (BM25 full-text by default, with optional semantic search, one arm per query), and provider-agnostic replay all sit on a single Lance-on-object-storage foundation.

The v1 surface includes: full CLI, HTTP+JSON and MCP transports, search over three Lance datasets, opt-in intfloat/multilingual-e5-small embeddings at FP16 weights (Metal on macOS, CUDA opt-in, CPU fallback), and local-FS / S3 / GCS / Azure backends through Lance's object_store integration.

Install

Linux, macOS, and Windows are supported.

Package Managers (macOS and Linux):

brew install tenequm/tap/pond              # Homebrew
nix profile add github:tenequm/pond#pond   # Nix
cargo install pond-db                      # crates.io (installs the `pond` command)

Build from source:

git clone https://github.com/tenequm/pond.git
cd pond
cargo install --path packages/pond

For CUDA acceleration on Linux:

cargo install --path packages/pond --features cuda

On macOS the Metal backend is selected automatically; on other systems the CPU fallback runs without extra features.

On Windows, the Homebrew/Nix packages do not apply:

winget install tenequm.pond

scoop bucket add tenequm https://github.com/tenequm/scoop-bucket
scoop install pond

cargo binstall pond-db

The winget manifest is in review at winget-pkgs, so winget install does not resolve yet - use Scoop until it merges. See the Windows notes for building from source, Defender, long paths, and WSL.

Usage

Sync and search

Set up storage, adapters, MCP registration, and an optional sync schedule in one pass (idempotent - re-run it any time to repair or update):

pond init

Then import sessions from local adapters, update indexes, and search:

pond sync
pond search "how did we wire up the OCC retry loop"

Run a server

pond serve                         # HTTP on 127.0.0.1:9797
pond serve --transport stdio       # MCP over stdio
pond mcp                           # alias for stdio MCP

Fetch and copy

Fetch a single session or message, or move a whole corpus:

pond get-session <id>
pond get-message <id>
pond copy --from local --to snapshot.pond
pond copy --from snapshot.pond --to local

Read-only SQL

Ask structured questions with read-only SQL (the same surface as the pond_sql MCP tool):

pond sql "SELECT project, count(*) FROM messages GROUP BY project ORDER BY 2 DESC"

Maintenance

Run maintenance on demand (sync folds indexes on every run):

pond optimize --only embed   # only when [embeddings].enabled = true
pond optimize --only index

Scheduled sync

Keep pond current automatically (launchd on macOS, systemd user timers or cron on Linux, Task Scheduler on Windows):

pond schedule start                # every 5m by default (--every 15m|1h|6h|1d)
pond schedule status
pond schedule logs

Status and introspection

pond status prints a per-table storage table, then indexes (text readiness, plus the semantic half only when embeddings are enabled), stored (sessions + messages), agents (source agents in the store), and this host's view of it: per-adapter sessions pending sync, the last sync's outcome (including a surfaced failure from a scheduled run), and the next scheduled run. pond status --hosts breaks a shared store down by ingest host; --include-subagents counts each subagent as its own agent. pond sync --dry-run previews what the next sync would read. pond search --explain returns Lance's analyze_plan output for each retrieval arm.

Remote storage

By default pond stores data locally under $XDG_DATA_HOME/pond. To use an object store, add credentials and switch the destination:

pond creds add                                                    # interactive: name, access key, hidden secret
pond storage use s3+https://nbg1.your-objectstorage.com/my-pond   # probe end-to-end, then flip [storage].path
pond storage check                                                # verify: parse, creds, conditional-put (OCC), write/read/delete

pond init --storage-path <url> configures a remote destination during setup and prompts for credentials inline when the destination is remote, so a bucket is one command. The s3+https://host/bucket form works for any S3-compatible store (Hetzner, R2, B2, MinIO); s3://, gs://, and az:// use the standard cloud SDK credential chain when no [creds.*] set matches. pond copy --from <local> --to <url> carries existing local data into the bucket - idempotent, never deletes the source, and on completion it rebuilds the destination indexes and verifies every row landed (exit 6 if any are missing or duplicated, so you never reconcile by hand). pond copy --verify-only --from <local> --to <url> runs that same check read-only, without copying. Full walkthrough: pond.locker.

Configuration

pond init walks through everything below interactively and enables the adapters it finds. pond sync only ingests already-enabled adapters - enabling one is an explicit step (pond adapters enable / pond adapters discover / pond init), never a side effect of sync. Config lives under $XDG_CONFIG_HOME/pond/. Every [adapters.<name>] block needs enabled = true to be active; sections without it (or with enabled = false) are skipped.

[adapters.claude-code]
enabled = true
path = "~/.claude/projects"

[adapters.codex-cli]
enabled = false                    # kept in config, skipped on `pond sync`
path = "~/.codex/sessions"

Search is BM25 full-text by default. Semantic search is opt-in and off unless you ask for it: with it off no pond process downloads or loads an embedding model, new messages get no vectors, and --mode vector is refused. Turn it on in config or with POND_EMBEDDINGS_ENABLED=true, then run pond optimize --only embed once to fill the backlog:

[embeddings]
enabled = true

Full detail, including what it costs and how mixed fleets behave, is in the configuration reference.

Supported harnesses

One adapter per harness, in pond adapters discovery order; Reads is the path pond init discovers and writes to [adapters.<name>].path. Last verified is the most recent capture or refresh date of the adapter's committed fixture (packages/pond/tests/fixtures/adapter/), the corpus its mapping is tested against. Adapters are maintained best-effort, and format drift is safe by design: unknown record shapes still ingest losslessly, malformed input surfaces as a typed error naming the file. Adding a harness is routine work - see Contributing.

Adapter Reads Last verified
claude-code Claude Code CLI, ~/.claude/projects 2026-08-14
claude-desktop-app Claude Desktop / Cowork local agent sessions 2026-05-13
claude-ai-export claude.ai data-export archive (manual --path) 2026-06-04
codex-cli OpenAI Codex CLI, ~/.codex/sessions 2026-05-13
opencode opencode, ~/.local/share/opencode (SQLite DB + legacy tree) 2026-07-14
openclaw openclaw, ~/.openclaw 2026-05-13
nanoclaw nanoclaw, ~/nanoclaw (the install root holding data/v2-sessions) 2026-05-14
hermes Hermes Agent, ~/.hermes (state.db per profile) 2026-07-23
pi-coding-agent pi, ~/.pi/agent/sessions 2026-08-06
oh-my-pi oh-my-pi, ~/.omp/agent/sessions (ingest-only) 2026-08-14
letta-code letta-code, ~/.letta/transcripts (a root relocated via LETTA_TRANSCRIPT_ROOT is configured as an explicit path) 2026-08-24
grok-build grok-build (xAI grok CLI), ~/.grok/sessions (a relocated GROK_HOME is configured as an explicit path) 2026-08-24

Verbosity

Root-level -v / -vv / -vvv raise the tracing level (info / debug / trace); -q / -qq lower it. The default surfaces warnings only. RUST_LOG overrides the CLI flag when set; POND_LOG is no longer honored.

Design

The full contract is in docs/spec.md. Key choices:

  • Lance direct, no wrapper. The lance-format/lance crates are the only storage and search engine. No lancedb, no parallel abstraction. Storage, indexing, OCC, schema evolution, blob columns, versioning, and time-travel are all Lance. The read-only pond sql surface is DataFusion planning over the same Lance datasets - a query escape hatch, not a second engine.
  • Canonical Session / Message / Part interlingua. Owned in pond, in the shape of Effect v4's Prompt-side Part union. This schema is pond's product; everything else is machinery around it.
  • Three Lance datasets (sessions, messages, parts). messages carries the nullable embedding (vector + embedding_model) alongside denormalized filter columns (source_agent / project / role / timestamp) for single-stage filter pushdown.
  • No-synthesis adapter seam. Adapters parse source records through extractor helpers that make "invent a value" a compile error - model-no-synthesis, model-schema-honesty, and adapter-provenance-required are structural, not review rules.
  • Index lifecycle decoupled from writes. Writes commit data (including embeddings, computed inline at ingest when embeddings are enabled) without folding the search indexes. pond sync runs index maintenance by default, and pond optimize --only index runs it on demand; Lance merges index results with a flat scan over unindexed fragments, so reads stay correct.
  • Single-arm retrieval. Each query runs one retriever - fts (BM25, the default) or vector (cosine, with a gentle recency tiebreaker, offered only when embeddings are enabled) - chosen per query; no server-side fusion. --sort-by recency returns newest-first. Results group to one summary per session, keyed on session_root.
  • Language-neutral full-text. Word-level simple tokenizer with English stemming (ascii-folding on); tokens the stemmer does not recognize pass through unchanged and stay exact-matchable, so pond indexes sessions in any language alike.
  • Two transports, one handler set. HTTP+JSON (axum) and MCP (rmcp) both dispatch into the same handlers. Wire ops: pond_search, pond_get_session, pond_get_message, pond_ingest. MCP additionally exposes the read-only pond_sql tool and the schema://pond, schema://pond-sql, and stats://pond resources.
  • Opaque-string multi-tenancy. Each tenant is a namespace string the integrator supplies; pond does not authenticate, authorize, or model identity. The object store's IAM is the storage boundary.
  • Encryption is operational. Bucket SSE plus filesystem encryption; pond holds no keys and adds no application-level crypto.

Roadmap

pond ships in small steps. This table lists the steps in order. Done steps stay in the table. The roadmap board holds the same items with their issues. React or comment on an issue to influence the order.

# Step Status
1 Lossless ingest from Claude Code and Codex into Lance, local or S3 v0.5
2 Single-arm search: BM25 or vector, one arm per query v0.10
3 Remote sync in under a minute; warm search in under a second v0.11-v0.12
4 Tool-call columns and read-only SQL over the corpus v0.13
5 Crash-safe local stores that self-heal on open v0.14.0
6 Eleven harnesses, pond resume into any client, MCP registry, Windows v0.14.11
7 BM25 becomes the default arm. Embeddings become opt-in. #164 v0.15.0
8 New adapters become routine: add-adapter playbook + conformance harness #172, letta-code #170, grok CLI #171 - twelve harnesses v0.15.1
9 Lance 10: count pushdown, date-filter zonemaps, faster index commits. #145 ⏭ Next
10 pond erase: the one sanctioned deletion. #45 ⏭ Next
11 Remote reads as fast as local reads. #165 ⏳ Later
12 Namespaces: keep work and personal sessions apart. #166 ⏳ Later
13 Redaction on copy, export, and resume. Never at ingest. #167 ⏳ Later
A capture daemon. pond reads what your harness already writes. ❌ Not planned
A hosted service that owns your bucket. Your storage stays yours. ❌ Not planned
Summaries or pruning of stored sessions. pond keeps the sessions. ❌ Not planned

Step 7 was based on data. Over 63 days, agents ran 1,126 searches against this archive. BM25 found the answer 61% of the time. Vector found it 37% of the time. Read the measurement.

This is a direction, not a contract. The order changes when the data changes.

References

The upstream schemas that shaped pond's canonical model are documented in docs/references/ (source URLs + why each matters; the vendored code itself is not redistributed). Real session captures live under packages/pond/tests/fixtures/adapter/.

Source Why it matters
Effect-TS/effect Effect v4 Prompt/Response Part unions. Pond's canonical types copy this shape.
sst/opencode Effect Schema canonical Part union; SDK types; storage schema.
kilo-org/kilocode OpenCode fork. Adds editorContext, plan-followup, kilocode-specific events.
badlogic/pi-mono pi-coding-agent leaf-cursor branching and cross-provider conformance test matrix.
open-telemetry/semantic-conventions-genai GenAI semantic conventions. Inspiration for shape overlap; pond does not derive from OTel.
packages/pond/tests/fixtures/adapter/ Session samples for thirteen source harnesses (claude_ai_export, claude_code, claude_desktop_app, claude_managed_agents, codex_cli, grok-build, hermes, letta-code, nanoclaw, oh-my-pi, openclaw, opencode, pi-coding-agent; real captures except the synthetic hermes state.db and the generated oh-my-pi corpus). Drives adapter design and serves as adapter test fixtures.

Contributing

Issues and pull requests are welcome. The most useful contributions right now:

  • An adapter for a harness pond does not read yet. The playbook is the add-adapter skill; the PR expectations are in CONTRIBUTING.md.
  • Spec feedback on docs/spec.md.
  • Pointers to additional reference schemas or session samples worth documenting under docs/references/.
  • Bug reports against the v1 surface (CLI verbs, wire ops, schema mismatches, OCC behavior, object-store backends).

For something bigger, the roadmap is the list. Comment on the issue before you start, so we agree on the scope. For other larger changes, open an issue first to discuss the direction. For security issues, see SECURITY.md.

Questions or feedback? Start a GitHub Discussion, or DM me on Telegram or X - I answer personally.

Links

License

Apache-2.0 (c) 2026 tenequm