Tokenectomy Razor π‘οΈ β Autonomous Context Surgery & Secret Redaction for AI Agents (M2M MCP Server)
π Interactive Web Playground & Live Architecture β
π¬ Need AST Auto-Healing, Test Rollback & Pro Engine? Tokenectomy Sentinel (Pro Tier) β
π Pair with Tokenectomy Git (OSS) for autonomous Git fix branches & PR creation!
π Telemetry Receipts (25K β 1M Lines): Check out the transparent Benchmark History repository.
Tokenectomy Razor π‘οΈ (Community / OSS Tier) is an agent-native Machine-to-Machine (M2M) MCP server built with Rust. Designed specifically as an autonomous background sub-cortex for AI coding agents (Claude Desktop, Cursor, Cline, Roo Code, Windsurf, Google Antigravity), Razor surgically scrubs 90%+ of internal framework noise (node_modules, site-packages, .cargo/registry) from error logs, auto-redacts sensitive credentials before cloud transmission, and provides sub-millisecond AI reverse proxyingβwith zero human babysitting.
ββββββββββββββββββββ
Agent Error Dump β TOKENECTOMY β Clean Agent Context
(38K tokens) ββββΊ β RAZOR π‘οΈ β ββββΊ (2K tokens) ββββΊ LLM Brain
β Community / OSS β
node_modules/ β π Smart Filter β Only YOUR code
site-packages/ β π‘οΈ Redact β + error message
.cargo/registry/ β πΎ Cache β + StackOverflow refs
ββββββββββββββββββββ
β¨ Key Features
- π Polyglot Trace Surgery β Natively parses stack traces and crashes across Rust, Python, Node.js/TypeScript/JSX, Golang, Java/Kotlin, C/C++ (ASan & GDB), and PHP (Laravel/Symfony), auto-filtering thousands of lines of framework dependency noise (
node_modules,site-packages,go/src,pkg/mod,.gradle,.m2,/usr/include,vendor). - π‘οΈ AI Gateway Reverse Proxy (
--proxy) β Intercepts OpenAI/Anthropic/Ollama API traffic locally (127.0.0.1:8080), surgically scrubbing prompt token waste and auto-redacting secrets with sub-millisecond latency before forwarding to upstream LLMs. - π Smart Framework Filter β Strips thousands of lines of noisy internal stack frames and preserves strictly the lines of code you wrote.
- π Stack Overflow Search β Silently queries StackExchange APIs and injects top community solutions into the AI's context.
- β‘ SHA-256 Response Cache β Identical errors hit local cache (24h TTL). Recurring CI/CD failures cost $0.00 in API calls.
- π‘οΈ Secret Redaction β Regex engine strips API keys, AWS secrets, JWTs, and database connection strings before any data leaves your machine (ReDoS-safe, linear-time).
- β‘ High-Throughput Stream Surgery β Zero-allocation linear-time $O(N)$ evaluation handling 250,000+ lines in ~330ms without memory bloat.
- π Path Traversal Protection β MCP file operations are strictly canonicalized and locked within your workspace.
- π€ MCP Server Mode β Full JSON-RPC 2.0 over stdio. Works with Claude Desktop, Cursor, VS Code, Google Antigravity, and any MCP-compatible client.
- π Multi-Provider β Supports OpenAI, Anthropic, and Ollama (100% offline mode).
π Verifiable Real-World Performance Benchmark
Every developer can verify the core performance claims directly on their physical machine:
| Feature Under Test | Tested Heavy Input | Real Measured Outcome | Status |
|---|---|---|---|
| Quarter-Million Log Redaction | 250,000 lines (24.44 MB) enterprise dump with DB URLs, API keys, JWTs | 333.49 ms (73.3 MB/sec, 749,652 lines/sec). 100% sanitized. | β Verified |
| ReDoS Immunity | 50,000-character malicious backtracking exploit string | 1.44 ms. Linear $O(N)$ evaluation, 100% ReDoS immune. | β Verified |
| High Concurrency Torture | 100 concurrent OS threads hammering redaction & extractor | 100/100 in 27.35 ms (7,312.7 ops/sec). Zero race conditions. | β Verified |
| Kernel Memory Footprint | Peak Resident Memory during 250,000-line stress test | 76.24 MB VmRSS via Linux /proc/self/status. Zero memory ballooning. |
β Verified |
π‘ Verify on your own hardware: Clone this repository and run the standalone benchmark:
π¬ Need enterprise workloads (1M+ lines, 250 threads, AST Smart Healer & test rollback)?
Check out Tokenectomy Sentinel (Pro Tier) on Gumroad ($9) β.
π¦ Installation
β‘ Install via Cargo (crates.io)
π¦ Build from Source
# Optional alias for backward compatibility:
βοΈ Install via Smithery (for Claude Desktop)
π M2M Agent Setup (1-Minute Integration)
Tokenectomy Razor is architected to run silently between your AI Coding Agent and your repository over JSON-RPC 2.0 stdio. You configure it once, and your agent autonomously invokes Tokenectomy Razor in the background during debugging and refactoring loopsβno manual copy-pasting or piping required.
# (or legacy alias: tkmy --mcp)
Claude Desktop
Add to ~/Library/Application Support/Claude/claude_desktop_config.json:
Cursor
Add to .cursor/mcp.json in your project root:
Cline / Roo Code / Windsurf / VS Code
Add to your MCP settings (settings.json or cline_mcp_settings.json):
Google Antigravity CLI
π€ Available M2M MCP Tools
| Tool | Autonomous Agent Role |
|---|---|
get_error_context |
Performs deep log surgery: strips framework noise, redacts secrets, extracts source context and git diff |
search_stack_overflow |
Searches Stack Overflow for a specific error (query is auto-sanitized of secrets) |
apply_code_patch |
Applies a code patch to a file by search-and-replace |
π Companion MCP Server: Tokenectomy Git
Close the autonomous loop from error diagnosis all the way to a published GitHub Pull Request! Pair Tokenectomy Razor with our official companion MCP server: Tokenectomy Git.
Together, they enable your AI coding assistant to:
- Scrub noisy error logs & redact credentials (
tokenectomy) - Generate an accurate fix patch
- Create a branch, commit files, and open a GitHub PR autonomously (
tokenectomy-git)
π‘οΈ AI Gateway Reverse Proxy Mode (Zero-Config Token Optimization)
Want token reduction without configuring MCP tools? Tokenectomy Razor can run as a local AI Reverse Proxy Gateway. It sits transparently between your IDE/agent and upstream LLM providers (OpenAI, Anthropic, Ollama, OpenRouter).
Whenever your agent makes an API call, Tokenectomy Razor intercepts the prompt payload, redacts sensitive credentials, and surgically purges internal framework noise before forwarding the requestβstreaming the LLM response back with sub-millisecond overhead.
# Start Gateway Proxy forwarding to OpenAI
# Or forward to a local Ollama instance
Connect Any Agent or Tool in 1 Line:
Point your agent's API base URL to localhost:
Works out-of-the-box with Cursor, Aider, Cline / Roo Code, Continue.dev, Open-Interpreter, and any OpenAI SDK client!
π Polyglot Ecosystem Support Matrix
Tokenectomy Razor features zero-allocation compiled regex trace parsers tailored for production backends:
| Language | Ecosystems & Frameworks | Filtered Framework Noise |
|---|---|---|
| Rust | tokio, actix-web, axum |
.cargo/registry, .rustup, target/debug/build |
| Python | Django, FastAPI, PyTorch |
site-packages, dist-packages, venv, __pycache__ |
| TypeScript / JS | Next.js, Vite, Express, NestJS |
node_modules, .next, dist, webpack internals |
| Golang | Goroutine panics, Gin, Fiber |
go/src (stdlib), go/pkg/mod, vendor |
| Java / Kotlin | Spring Boot, Quarkus, JVM exceptions |
.m2/repository, .gradle/caches, org.springframework |
| C / C++ | GDB backtraces, AddressSanitizer (ASan) | /usr/include, /usr/lib, vcpkg_installed |
| PHP | Laravel, Symfony, Fatal errors |
vendor/composer, vendor/symfony, vendor/laravel |
π οΈ Standalone / Local CLI Mode (Optional)
While Tokenectomy Razor is architected for autonomous machine-to-machine agent operation, it also provides a standalone CLI binary (razor, backwards-compatible with tkmy) if you want to pipe logs in CI/CD pipelines, local shell scripts, or manual debugging:
Pipe errors directly
|
|
|
# Or using legacy alias:
|
Read from a log file
Advanced options
π§ Configuration
Create ~/.tokenectomy.toml:
= "openai" # openai | anthropic | ollama | mock
= "sk-..."
= "sk-ant-..."
= "http://localhost:11434"
= 10
= 10000
π Security
- No secrets leave your machine. Regex engine redacts API keys, JWTs, AWS credentials, and database URLs before any data is sent to an LLM.
- No path traversal. MCP file operations are canonicalized and locked to CWD.
- No stdin bombs. Input is capped at 10MB (CLI) / 50MB (MCP) via
.take(). - No weak hashing. Cache uses
sha2::Sha256, neverDefaultHasher.
ποΈ Architecture
src/
βββ lib.rs # Core library interface
βββ app.rs # CLI application runner & REPL
βββ main.rs # `tokenectomy` binary entry point
βββ bin/ # Standalone binary aliases (`razor`, `tokenectomy-razor`)
βββ mcp.rs # JSON-RPC 2.0 over stdio MCP server
βββ proxy.rs # AI Gateway Reverse Proxy (TCP socket prompt compressor)
βββ extractor/ # Polyglot trace parsers (Rust, Python, JS/TS, Go, Java, C++, PHP)
βββ provider/ # AI backends (OpenAI, Anthropic, Ollama, Mock test provider)
βββ redact.rs # Secret redaction (linear-time regex, ReDoS-safe)
βββ cache.rs # SHA-256 response cache (24h TTL, 0700 perms)
βββ search.rs # Stack Overflow API integration
βββ git.rs # Recent git diff extraction
π§ Bundled Open Source Agent Skills
Tokenectomy OSS includes 2 native Antigravity & Coding Agent Skills in .agents/skills/ to elevate your AI assistant's engineering discipline:
| Skill | Description | Location |
|---|---|---|
adversary-bug-hunter |
Red-team fuzzer that stress-tests edge cases, catches unhandled unwraps/ReDoS, and hardens code. | .agents/skills/adversary-bug-hunter/ |
spec-first-architect |
Enforces strict Test-Driven Development (TDD) & state machine invariants to eliminate AI hallucinations. | .agents/skills/spec-first-architect/ |
π vs π¬ β Razor (OSS) vs Sentinel (Pro)
| Feature | π‘οΈ Razor (OSS) | π¬ Sentinel (Pro) |
|---|---|---|
| Smart Framework Filter | β | β |
| Polyglot Trace Surgery (7 Languages) | β | β |
Local Reverse Proxy Gateway (--proxy) |
β | β |
| Stack Overflow Search | β | β |
| SHA-256 Response Cache | β | β |
| Secret Redaction (ReDoS-safe) | β | β |
| MCP Server Mode (JSON-RPC) | β | β |
| Multi-Provider (OpenAI, Claude, Ollama) | β | β |
| Bundled Agent Skills | 2 Skills (Spec TDD & Fuzzer) | Full 4 Skills Suite |
| Anti-Hardcode Secret Shield | β | β |
| AST Syntax Validation (Tree-sitter) | β | β |
| AST Smart Healer (auto-syntax fix) | β | β |
| Code Integrity Guard (Anti-Halu & Anti-Ngide) | β | β |
| Test Verification Loop (Auto-Rollback) | β | β |
| Atomic Multi-File Transactions | β | β |
Time Machine CAS Undo Engine (--undo) |
β | β |
| True Ectomy Engine (99% reduction) | β | β |
| DB Inspector & Docker Diagnostics | β | β |
Reproducible --benchmark Mode |
β | β |
π Get Tokenectomy Sentinel on Gumroad ($9) β
π License
MIT β see LICENSE for details.
Built with π¦ Rust for maximum performance, strict memory safety, and uncompromising security.