// Add to ~/.cursor/mcp.json or claude_desktop_config.json
β‘ The 30-Second Surgery
When an autonomous coding agent runs a failing build or test, the terminal dumps tens of thousands of tokens of internal framework stack frames and potentially leaks production secrets directly into the LLM context window.
Raw Terminal Crash (45,820 tokens + Leaked Secrets)
β
βΌ <0.2ms Zero-Allocation Rust DFA Excision
[Redact Secrets Locally] βββΊ [Filter Framework Frames] βββΊ [Extract Source Context]
β
βΌ
Sanitized Agent Context (118 tokens β’ Zero Secrets β’ Sub-millisecond)
Before & After Comparison
β Before: Raw Crash Dump (45,820 Tokens Ingested)
TypeError: Cannot read properties of undefined (reading 'digest')
at Object.<anon> (/node_modules/next/bundle5.js:142:31)
at __webpack_require__ (/node_modules/next/bundle5.js:198:12)
at Object.execute (/node_modules/next/dev-server.js:412:19)
at processTicksAndRejections (task_queues:95:5)
Database connection failed: postgresql://admin:super_secret_password@db.prod.internal:5432/primary
API key leaked: sk-ant-api03-abcdef1234567890abcdef1234567890
[... 480 internal dependency frames flooding LLM context ...]
β After: Tokenectomy Razor (118 Tokens β’ <0.2ms β’ Zero Secrets)
src/components/Header.tsx:42:15 - SyntaxError
42 | const user = useSession( ;
| ^ Expected ')'
π‘οΈ [CONNECTION_STRING_REDACTED]
π‘οΈ [REDACTED] ANTHROPIC_API_KEY=[REDACTED_SECRET_KEY]
Result: 99.7% context token reduction, zero credential leakage, prompt cache preserved.
π¬ Verifiable Benchmarks
All performance claims are hardware-grounded and independently reproducible on physical hardware (measured on 10-Core Intel Core i5-1235U @ 15W running Arch Linux, Kernel 6.13):
$ cargo test --release --test stress_benchmark -- --nocapture
=====================================================================================
π§ͺ TOKENECTOMY OSS VERIFIABLE HEAVY STRESS BENCHMARK (100% REPRODUCIBLE IN OSS)
Hardware: 10-Core / 12-Thread Intel Core i5-1235U | OS: Arch Linux | Kernel Telemetry Active
Initial Baseline Process Memory (VmRSS): 3.45 MB
=====================================================================================
π₯ [TEST 1/3] QUARTER-MILLION LINES LOG REDACTION TORTURE (250,000 LINES / 25MB+ BUFFER)
βββ Buffer Size: 24.44 MB (250000 lines)
βββ Redaction Latency: 471.05ms (51.9 MB/sec)
βββ Line Throughput: 530,735 lines/sec
βββ Peak Memory (VmRSS): 76.05 MB (Delta: +72.60 MB)
βββ Status: β
PASSED (100% of 250,000 lines sanitized, zero memory balloon)
π₯ [TEST 2/3] REDOS CATASTROPHIC BACKTRACKING TORTURE (50,000 CHARS PAYLOAD)
βββ Attack Payload Size: 50,082 characters
βββ Execution Latency: 1.165 ms
βββ Status: β
PASSED (Linear O(N) evaluation, 100% ReDoS Immune)
π₯ [TEST 3/3] HIGH-CONCURRENCY TORTURE (100 PARALLEL OS THREADS)
βββ Thread Concurrency: 100 concurrent OS threads
βββ Successful Operations: 100/100 (100.0%)
βββ Total Elapsed: 11.31ms
βββ Concurrency Throughput: 17,688 ops/sec
βββ Final VmRSS: 78.99 MB
βββ Status: β
PASSED (Zero race condition, zero deadlock)
=====================================================================================
π TOKENECTOMY OSS STRESS BENCHMARK: 3/3 PASSED (100% GREEN)
Total Suite Duration: 580.83ms
Bounded Final VmRSS: 78.99 MB
=====================================================================================
test test_oss_heavy_stress_benchmark ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.58s
| Benchmark Target | Workload Under Test | Verified Measurement | Result |
|---|---|---|---|
| Log Redaction Throughput | 250,000 lines (24.44 MB) enterprise dump with API keys & connection URIs | 530,735 lines/sec (471.0 ms, 51.9 MB/s) | Pass |
| ReDoS Immunity | 50,000-character pathological backtracking regex payload | 1.16 ms (Strict Linear $O(N)$ Evaluation) | Pass |
| Thread Concurrency | 100 concurrent OS threads executing simultaneous redaction | 17,688 ops/sec (100/100 completed in 11.31 ms) | Pass |
| Memory Footprint | Peak Resident Memory during 250k-line continuous stress test | 76.05 MB VmRSS via /proc/self/status |
Pass |
| Release Test Suite | Full integration test matrix across extractors, filters, and analyzers | 57 / 57 Verified Green (Zero panics, zero leaks) | Pass |
π‘οΈ Automated Redaction & Secret Sanitization Benchmark
Automated evaluation across 12 polyglot crash traces (Rust, Python, TypeScript, Go, YAML) containing 22 ground-truth credentials and clean negative controls. Evaluated head-to-head against Gitleaks v8.30.1.
1. Per-Category Precision, Recall & F1-Score
| Secret Category | Ground Truth | Razor Recall | Razor F1 | Gitleaks Recall | Gitleaks F1 | Sanitization Advantage |
|---|---|---|---|---|---|---|
Anthropic Claude API Key (sk-ant-...) |
1 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
AWS Access Key ID (AKIA...) |
1 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
| AWS Secret Access Key | 1 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
| Database URI (PostgreSQL, MySQL, Redis, Mongo) | 4 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
| Generic Passwords / Auth Secrets (YAML/JSON) | 3 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
GitHub Personal Access Token (ghp_...) |
1 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
GitLab Personal Access Token (glpat-...) |
1 | 100.0% | 100.0% | 100.0% | 100.0% | Parity (100% caught) |
HuggingFace API Token (hf_...) |
1 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
| JSON Web Token (RFC 7519 / Truncated) | 2 | 100.0% | 100.0% | 100.0% | 100.0% | Parity (100% caught) |
npm Registry Access Token (npm_...) |
1 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
OpenAI API Key (sk-..., sk-proj-...) |
1 | 100.0% | 100.0% | 100.0% | 100.0% | Parity (100% caught) |
| PEM Private RSA Key Block | 1 | 100.0% | 100.0% | 100.0% | 100.0% | Parity (100% caught) |
PyPI Package Upload Token (pypi-AgEI...) |
1 | 100.0% | 100.0% | 100.0% | 100.0% | Parity (100% caught) |
SendGrid API Key (SG...) |
1 | 100.0% | 100.0% | 0.0% | 0.0% | +100% Recall (M2M zero-leak) |
Slack Bot/User Token (xoxb-...) |
1 | 100.0% | 100.0% | 100.0% | 100.0% | Parity (100% caught) |
Stripe Live/Test Secret Key (sk_live_...) |
1 | 100.0% | 100.0% | 100.0% | 100.0% | Parity (100% caught) |
2. Head-to-Head Performance & Architectural Summary
| Dimension | Tokenectomy Razor (--scrub) |
Gitleaks v8.30.1 | Architectural Rationale |
|---|---|---|---|
| Overall Secret Recall | 100.0% (22/22) | 36.4% (8/22) | Razor captures unquoted URIs, DB ports & AI keys missed by diff rules |
| Overall Precision | 100.0% (0 False Positives) | 88.9% | Zero false triggers on compiler errors & minified traces |
| Overall F1-Score | 100.0% | 51.6% | Comprehensive coverage engineered specifically for crash context |
| Execution Engine | Zero-allocation Rust DFA ($O(N)$) | Go regex scanner + Git tree crawler | Sub-millisecond latency for agent streaming backtraces |
| ReDoS Immunity | Guaranteed Linear Time ($O(N)$) | Engine dependent | Immune to catastrophic backtracking on massive dumps |
| Sanitization Action | Inline token redaction ([KEY_REDACTED]) |
Warning log only (No scrub) | Directly sanitizes text before ingestion by LLM cortex |
3. Token Reduction & LLM Context Savings (tiktoken cl100k_base)
| Metric | Measured Value | Operational Impact for AI Coding Agents |
|---|---|---|
| Mean Token Reduction | 41.67% | Consistently shrinks raw crash trace token footprint |
| Median Reduction (P50) | 42.95% | Typical credential and connection dump reduction |
| 90th Percentile (P90) | 61.42% | Eliminates long multi-line keys and credentials |
| Min / Max Spread | 0.00% β 81.36% | 0% on clean negative controls (zero distortion), up to 81.4% on leaks |
| Total Tokens Preserved / Saved | 920 tokens (44.02% net) | Prevents context window saturation and reduces LLM billing |
π Quick Start
1. Model Context Protocol (MCP) Setup
Tokenectomy Razor operates natively over JSON-RPC 2.0 stdio, compliant with the official Model Context Protocol specification.
Cursor Composer
Add to .cursor/mcp.json in your workspace root:
Claude Desktop
Add to claude_desktop_config.json:
- macOS:
~/Library/Application Support/Claude/claude_desktop_config.json - Windows:
%APPDATA%\Claude\claude_desktop_config.json - Linux:
~/.config/Claude/claude_desktop_config.json
Windsurf (Codeium)
Add to ~/.codeium/windsurf/mcp_config.json:
VS Code (Cline / Roo Code)
In Cline or Roo Code settings (cline_mcp_settings.json):
Google Antigravity CLI
2. Standalone CLI & Terminal Piping
When debugging or piping terminal output directly:
# Pipe terminal test failures through the surgical redactor:
|
# Sanitize a specific raw log file:
# Offline local air-gapped mode (zero external network calls):
|
3. AI Gateway Reverse Proxy (--proxy)
Tokenectomy Razor can operate as a high-throughput local HTTP reverse proxy on 127.0.0.1:8080. It intercepts outbound prompt streams, performs real-time token excision and credential sanitization, and forwards clean requests upstream to OpenAI, Anthropic, or Ollama.
# Start local gateway proxy forwarding to OpenAI:
# Start local gateway forwarding to Ollama:
Point any standard SDK client to the local proxy:
FinOps Economics Dashboard
Open http://127.0.0.1:8080/dashboard in any browser to monitor real-time token savings, dollar savings (blended LLM pricing), total requests, and active security redactions.
π οΈ Exposed MCP Tools
Tokenectomy Razor complies with Glama Grade A Tool Definition Quality Score (TDQS) with explicit parameter boundaries:
| Tool Name | Capability Description |
|---|---|
get_error_context |
Performs trace surgery on error dumps, removes framework noise, redacts credentials, and extracts relevant local source context bounded to the workspace. |
analyze_code |
Performs static AST code analysis to detect resource leaks, unclosed handles, and syntax vulnerabilities with bounded execution limits and precise LSP UTF-16 coordinates. |
apply_code_patch |
Applies atomic file modifications with post-write language syntax verification (cargo check, py_compile, node --check) and automated rollback on failure. |
search_stack_overflow |
Queries Stack Exchange API for relevant error signatures using sanitized, redacted search terms. |
π Supported Polyglot Ecosystems
| Language | Frameworks Supported | Excluded Framework Internals |
|---|---|---|
| Rust | Tokio, Actix-web, Axum | .cargo/registry, .rustup, target/debug/build |
| TypeScript / JS | Next.js, Express, NestJS, Vite | node_modules, .next, dist, webpack internals |
| Python | Django, FastAPI, Flask, PyTorch | site-packages, dist-packages, venv, __pycache__ |
| Golang | Gin, Fiber, Stdlib Panics | go/src (stdlib), go/pkg/mod, vendor |
| Java / Kotlin | Spring Boot 3, Tomcat, Netty | .m2/repository, .gradle/caches, internal bytecode |
| C / C++ | AddressSanitizer, GDB / LLDB | /usr/include, /usr/lib, vcpkg_installed |
| PHP | Laravel, Symfony | vendor/composer, vendor/symfony |
π¦ Installation Options
Method 1: Instant via npx (Zero Toolchain Setup)
Method 2: Cargo (crates.io)
Method 3: Precompiled Standalone Binaries
Zero-dependency, standalone release binaries available on GitHub Releases:
- Linux:
x86_64-unknown-linux-gnu,x86_64-unknown-linux-musl,aarch64-unknown-linux-gnu - macOS:
aarch64-apple-darwin(Apple Silicon M1/M2/M3/M4),x86_64-apple-darwin(Intel) - Windows:
x86_64-pc-windows-msvc.exe
Method 4: Multi-Arch Docker Container (GHCR)
βοΈ Edition Comparison
| Capability | Razor (Community OSS) | Sentinel (Commercial Tier) |
|---|---|---|
| Polyglot Stack Trace Surgery | Yes (4 Languages) | Yes (All 7 Languages) |
| O(N) ReDoS-Safe Secret Redaction | Yes | Yes |
| JSON-RPC 2.0 MCP Server | Yes | Yes |
AI Gateway Reverse Proxy (--proxy) |
Yes | Yes |
| SHA-256 Idempotency Cache (24h TTL) | Yes | Yes |
| FinOps Metrics Dashboard | Yes | Yes |
| Tree-sitter AST Syntax Healing | β | Yes |
| Anti-Hallucination Scope Guard | β | Yes |
| Automated Test Rollback (0 Dirty Diff) | β | Yes |
| Multi-File Atomic Transactions | β | Yes |
Time Machine Undo Engine (--undo) |
β | Yes |
| Autonomous Healing State Machine | β | Yes |
Need Enterprise AST Self-Healing? Explore the Sentinel Tier
πΊοΈ Roadmap & Milestones
| Milestone / Capability | Status | Target |
|---|---|---|
| Core Polyglot Log Surgery & $O(N)$ ReDoS Redaction | β Complete | v1.0.0 |
AI Gateway Reverse Proxy (--proxy) & Idempotency Cache |
β Complete | v1.1.0 |
| Multi-arch Docker & GitHub Actions Marketplace Action | β Complete | v1.1.3 |
Static AST Analysis Engine (analyze_code) & UTF-16 LSP |
β Complete | v1.1.5 |
| Glama.ai Tool Definition Quality Score (TDQS Grade A) | β Complete | v1.1.5 |
| Precompiled Standalone Binaries (Linux, macOS, Windows) | β Complete | v1.1.6 |
Official MCP Registry Listing (io.github.Tokenectomy-Labs/razor) |
β Complete | v1.1.7 |
mcpservers.org Directory Listing |
β Complete | v1.1.7 |
| Java/Kotlin (Spring Boot 3) & C/C++ (ASan) Extractors | β Complete | v1.2.0 |
awesome-mcp-servers Community Catalog Listing |
π In Progress | v1.2.0 |
User-defined custom redaction patterns (~/.tokenectomy.toml) |
π Planned | v1.3.0 |
| Native VS Code & JetBrains companion extensions | π Planned | v1.4.0 |
| Server-Sent Events (SSE) remote MCP transport | π Planned | v1.4.0 |
π Security & Invariants
- Zero-Knowledge Architecture: All parsing, filtering, and secret redaction execute on physical local hardware. No logs are ever transmitted to third-party telemetry servers.
- Strict Linear $O(N)$ ReDoS Immunity: All pattern matchers utilize finite automaton evaluation with linear time guarantees, repelling catastrophic backtracking attacks.
- Path Traversal Boundary Isolation: File operations are strictly locked within the active workspace root (
CWD). Path traversals (../) and unauthorized symlinks are blocked. - Safe Rust Implementation: Core execution paths enforce safe Rust memory guarantees with bounded stream readers (
.take()) preventing resource exhaustion.
For vulnerability disclosures, please review our Security Policy.
π€ Community & Resources
- π Official Website
- π Documentation
- ποΈ System Architecture
- π Changelog
- π€ Contributing Guidelines
- π Security Policy