tokensave 1.2.1

Why tokensave?

When Claude Code works on a complex task, it spawns Explore agents that scan your codebase using grep, glob, and file reads. Every tool call consumes tokens.

tokensave gives Claude a pre-indexed semantic knowledge graph. Instead of scanning files, Claude queries the graph instantly — fewer API calls, less token usage, same code understanding.

How It Works

┌──────────────────────────────────────────────────────────────┐
│  Claude Code                                                 │
│                                                              │
│  "Implement user authentication"                             │
│        │                                                     │
│        ▼                                                     │
│  ┌─────────────────┐       ┌─────────────────┐               │
│  │  Explore Agent  │ ───── │  Explore Agent  │               │
│  └────────┬────────┘       └─────────┬───────┘               │
└───────────┼──────────────────────────┼───────────────────────┘
            │                          │
            ▼                          ▼
┌──────────────────────────────────────────────────────────────┐
│  tokensave MCP Server                                        │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐           │
│  │   Search    │  │   Callers   │  │   Context   │           │
│  │   "auth"    │  │  "login()"  │  │   for task  │           │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘           │
│         └────────────────┼────────────────┘                  │
│                          ▼                                   │
│              ┌───────────────────────┐                       │
│              │   libSQL Graph DB     │                       │
│              │   • Instant lookups   │                       │
│              │   • FTS5 search       │                       │
│              │   • Vector embeddings │                       │
│              └───────────────────────┘                       │
└──────────────────────────────────────────────────────────────┘

Without tokensave: Explore agents use grep, glob, and Read to scan files — many API calls, high token usage.

With tokensave: Agents query the graph via MCP tools — instant results, local processing, fewer tokens.

Key Features

Quick Start

1. Install the binary

Cargo (any platform):

cargo install tokensave

Homebrew (macOS):

brew install aovestdipaperino/tap/tokensave

From source:

git clone https://github.com/aovestdipaperino/tokensave.git
cd tokensave
cargo install --path .

2. Configure Claude Code

Run the built-in installer — no scripts, no jq, works on macOS/Linux/Windows:

tokensave claude-install

This single command:

• Registers tokensave as an MCP server in ~/.claude/settings.json
• Adds a native PreToolUse hook that blocks Explore agents in favor of tokensave
• Adds tool permissions so Claude can call all 9 tokensave tools without prompting
• Appends rules to ~/.claude/CLAUDE.md that instruct Claude to prefer tokensave over file reads

All changes are idempotent — safe to run again after upgrading.

3. Index your project

cd /path/to/your/project
tokensave sync

This creates a .tokensave/ directory with the knowledge graph database. Subsequent runs are incremental — only changed files are re-indexed.

MCP server

{
  "mcpServers": {
    "tokensave": {
      "command": "/path/to/tokensave",
      "args": ["serve"]
    }
  }
}

Tool permissions

{
  "permissions": {
    "allow": [
      "mcp__tokensave__tokensave_affected",
      "mcp__tokensave__tokensave_callees",
      "mcp__tokensave__tokensave_callers",
      "mcp__tokensave__tokensave_context",
      "mcp__tokensave__tokensave_files",
      "mcp__tokensave__tokensave_impact",
      "mcp__tokensave__tokensave_node",
      "mcp__tokensave__tokensave_search",
      "mcp__tokensave__tokensave_status"
    ]
  }
}

PreToolUse hook

The hook runs tokensave hook-pre-tool-use — a native Rust command (no bash or jq required). It intercepts Agent tool calls and blocks Explore agents and exploration-style prompts, redirecting Claude to use tokensave MCP tools instead.

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Agent",
        "hooks": [
          {
            "type": "command",
            "command": "/path/to/tokensave hook-pre-tool-use"
          }
        ]
      }
    ]
  }
}

CLAUDE.md rules

Appends instructions to ~/.claude/CLAUDE.md that tell Claude to use tokensave tools before reaching for Explore agents or raw file reads.

CLI Usage

tokensave sync [path]            # Sync (creates index if missing, incremental by default)
tokensave sync --force [path]    # Force a full re-index
tokensave status [path]          # Show statistics
tokensave query <search> [path]  # Search symbols
tokensave files [--filter dir] [--pattern glob] [--json]   # List indexed files
tokensave affected <files...> [--stdin] [--depth N]        # Find affected test files
tokensave claude-install         # Configure Claude Code integration
tokensave serve                  # Start MCP server

tokensave files

List all indexed files, optionally filtering by directory or glob pattern.

tokensave files                           # List all indexed files
tokensave files --filter src/mcp          # Only files under src/mcp/
tokensave files --pattern "**/*.rs"       # Only Rust files
tokensave files --json                    # Machine-readable output

tokensave affected

Find test files affected by source file changes. Uses BFS through the file dependency graph to discover impacted tests. Pipe from git diff for CI integration.

tokensave affected src/main.rs src/db/connection.rs   # Explicit file list
git diff --name-only HEAD~1 | tokensave affected --stdin   # From git diff
tokensave affected --stdin --depth 3 --json < changed.txt  # Custom depth, JSON output
tokensave affected src/lib.rs --filter "*_test.rs"    # Custom test pattern

Auto-sync on commit (optional)

Keep the index up to date automatically by running tokensave sync after every successful git commit. The repo includes a post-commit hook that does this in the background.

Global (all repos):

# Set a global hooks directory (skip if you already have one)
git config --global core.hooksPath ~/.git-hooks
mkdir -p ~/.git-hooks

# Install the hook
cp scripts/post-commit ~/.git-hooks/post-commit
chmod +x ~/.git-hooks/post-commit

Per-repo:

cp scripts/post-commit .git/hooks/post-commit
chmod +x .git/hooks/post-commit

The hook checks for both the tokensave binary and a .tokensave/ directory before running, so it is a no-op in repos that haven't been indexed.

MCP Tools Reference

These tools are exposed via the MCP server and available to Claude Code when .tokensave/ exists in the project.

tokensave_context

Get relevant code context for a task using semantic search and graph traversal.

• task (string, required): The task description
• max_nodes (number, optional): Maximum number of nodes to include (default: 20)

Returns structured code context with entry points, related symbols, and code snippets.

tokensave_search

Search for symbols by name in the codebase.

• query (string, required): Symbol name to search for
• kind (string, optional): Filter by node kind (function, class, method, etc.)
• limit (number, optional): Maximum results (default: 10)

Returns matching symbols with locations and signatures.

tokensave_callers / tokensave_callees

Find functions that call a symbol, or functions called by a symbol.

• symbol (string, required): The symbol to analyze
• depth (number, optional): Traversal depth (default: 1)
• limit (number, optional): Maximum results (default: 20)

Returns related symbols with relationship types.

tokensave_impact

Analyze the impact of changing a symbol. Returns all symbols affected by modifications.

• symbol (string, required): The symbol to analyze
• max_depth (number, optional): Maximum traversal depth (default: 3)

Returns impact map showing affected symbols and their relationships.

tokensave_node

Get detailed information about a specific symbol including source code.

• symbol (string, required): The symbol name
• file (string, optional): Filter by file path

Returns complete symbol details with source code, location, and relationships.

tokensave_files

List indexed project files. Use for file/folder exploration without reading file contents.

• path (string, optional): Filter to files under this directory
• pattern (string, optional): Filter files matching a glob pattern (e.g. **/*.rs)
• format (string, optional): Output format — flat or grouped (default: grouped)

Returns file listing with symbol counts, grouped by directory.

tokensave_affected

Find test files affected by changed source files. BFS through the file dependency graph to discover impacted tests.

• files (array of strings, required): Changed file paths to analyze
• depth (number, optional): Maximum dependency traversal depth (default: 5)
• filter (string, optional): Custom glob pattern for test files (default: common test patterns)

Returns the list of affected test files and count.

tokensave_status

Get index status and project statistics. Returns index metadata, symbol counts, language distribution, and pending changes.

How It Works with Claude Code

Once configured, Claude Code automatically uses tokensave instead of reading raw files when it needs to understand your codebase. The three layers reinforce each other:

The result: Claude gets the same code understanding with far fewer tokens. A typical Explore agent reads 20-50 files; tokensave returns the relevant symbols, relationships, and code snippets from its pre-built index.

How It Works (Technical)

1. Extraction

tokensave uses language-specific Tree-sitter grammars (native Rust bindings) to extract:

• Function and class definitions
• Variable and type declarations
• Import and export statements
• Method calls and references

2. Storage

Extracted symbols are stored in a local libSQL (Turso) database with:

• Symbol metadata (name, kind, location, signature)
• File information and language classification
• FTS5 full-text search index
• Vector embeddings for semantic search

3. Reference Resolution

The system resolves references between symbols:

• Import chains
• Function calls
• Type relationships
• Cross-file dependencies

4. Graph Queries

The graph supports complex queries:

• Find callers/callees at configurable depth
• Trace impact of changes across the codebase
• Build contextual symbol sets for a given task
• Semantic search via vector embeddings

Supported Languages

Troubleshooting

"tokensave not initialized"

The .tokensave/ directory doesn't exist in your project.

tokensave sync

MCP server not connecting

Claude Code doesn't see tokensave tools.

1. Ensure ~/.claude/settings.json includes the tokensave MCP server config
2. Restart Claude Code completely
3. Check that tokensave is in your PATH: which tokensave

Missing symbols in search

Some symbols aren't showing up in search results.

• Run tokensave sync to update the index
• Check that the language is supported (see table above)
• Verify the file isn't excluded by .gitignore

Indexing is slow

Large projects take longer on the first full index.

• Subsequent runs use incremental sync and are much faster
• Use tokensave sync (not --force) for day-to-day updates
• The post-commit hook runs in the background to avoid blocking

Origin

This project is a Rust port of the original CodeGraph TypeScript implementation by @colbymchenry. The port maintains the same architecture and MCP tool interface while leveraging Rust for performance and native tree-sitter bindings.

Building

cargo build --release
cargo test
cargo clippy --all

License

MIT License — see LICENSE for details.