# Tool output compression
[Feature docs index](README.md) · [Repository README](../../README.md)
## Purpose
Reduce provider-visible tokens for high-volume tool results with deterministic summaries while preserving raw local/session tool output.
## Enable
Disabled by default. Opt in through `~/.magi-code/settings.json`:
```json
{
"tools": {
"output_compression": { "enabled": true }
}
}
```
No slash command, shell proxy, RTK dependency, or per-rule setting exists for this feature.
## Scope
| Provider continuation payload | Curated summary for recognized `bash` commands when enabled. |
| Local terminal/TUI output | Original raw `ToolResult` remains displayed through existing output paths. |
| JSONL sessions | Original raw `ToolResult` is recorded before provider compression. |
| Hooks | Hook payloads and diagnostics use normal raw tool result behavior; compression does not change hook policy. |
| Non-`bash` tools | Always pass through unchanged. |
| Unrecognized `bash` commands | Pass through unchanged. |
| Compound shell commands | Pass through unchanged for unsafe operators (`&&`, `||`, `;`, background `&`, file redirects, `<`, `$()`, backticks, newline, carriage return). Exact fd redirects `2>&1` / `1>&2` and safe filter pipes are allowed. |
Compression changes what next provider request sees. It does not mutate tool execution result, success flag, local transcript, or session artifact already storing/displaying raw output.
## Supported first-slice rules
| `bash.git_status` | `git status` or `git --no-pager status` plus allowed short/branch args: `--short`, `--porcelain`, `--porcelain=v1`, `-sb`, `--branch` | Changed-file count, untracked count, branch/ahead-behind line when present, first capped status lines. |
| `bash.git_diff` | `git diff ...` or `git --no-pager diff ...` with no unsafe shell operators | Files touched from `diff --git`, hunk count, insertion/deletion counts, first capped file list, capped stat lines when present. |
| `bash.git_log` | `git log ... --oneline ...` or `git --no-pager log ... --oneline ...` with exact `--oneline` token | Non-empty commit line count and first 15 oneline subjects. |
| `bash.cargo_check` | `cargo check ...` with no unsafe shell operators | Exit code, warning/error counts, compiler error codes, final cargo status, first capped diagnostics. |
| `bash.cargo_test` | `cargo test ...` with no unsafe shell operators | Exit code, warning/error counts, compiler error codes, `test result:` lines, failing test names, first capped failure/panic context. |
Recognition uses conservative whitespace tokenization. Exact `2>&1` and `1>&2` fd redirect tokens are ignored for base command classification. Literal pipes are allowed only when every target starts with one whitelisted filter command: `grep`, `rg`, `tail`, `head`, `sed`, `cat`, `wc`, `sort`, `uniq`, `cut`, `tr`. Unsafe pipe targets such as `awk`, `xargs`, and `tee` pass through unchanged. If classification is uncertain, magi-code sends original output to provider.
## Provider-visible format
When compression applies, provider receives deterministic text:
```text
[tool_output_compression]
tool: bash
rule: bash.cargo_check
command: cargo check --all-targets
success: false
exit_code: 101
raw_stdout_bytes: 0
raw_stderr_bytes: 34812
stdout_truncated: false
stderr_truncated: false
compression: curated
exit_code: 101
warnings: 1
errors: 2
compiler_error_codes: E0425
final_status: error: could not compile `demo`
diagnostics:
- error[E0425]: cannot find value `x` in this scope
- warning: unused import: `Foo`
- error: could not compile `demo`
```
Header fields preserve operational context:
| Field | Meaning |
| --- | --- |
| `tool` | Tool name; currently always `bash` when compression applies. |
| `rule` | Curated rule id used to summarize output. |
| `command` | Original trimmed command text from tool call arguments. |
| `success` | Original tool success flag. |
| `exit_code` | Bash process exit code, or `null` when unavailable. |
| `raw_stdout_bytes` / `raw_stderr_bytes` | Byte counts from raw captured streams. |
| `stdout_truncated` / `stderr_truncated` | Raw stream truncation flags from bash metadata. |
| `compression` | Constant marker: `curated`. |
## Examples
### `git status --short --branch`
Raw stdout:
```text
## main...origin/main [ahead 1]
M src/lib.rs
?? notes.md
```
Provider-visible summary:
```text
[tool_output_compression]
tool: bash
rule: bash.git_status
command: git status --short --branch
success: true
exit_code: 0
raw_stdout_bytes: 61
raw_stderr_bytes: 0
stdout_truncated: false
stderr_truncated: false
compression: curated
changed_files: 2
untracked_files: 1
branch: ## main...origin/main [ahead 1]
status_lines:
- M src/lib.rs
- ?? notes.md
```
### `git diff -- src/lib.rs`
Summary counts files, hunks, insertions, deletions, and first capped files/stat lines. It does not attempt semantic diff interpretation.
`git --no-pager diff` is recognized the same way. `git diff | cat` is also recognized because `cat` is a safe filter and summary uses already-piped captured output.
### `git log --oneline -20`
Provider-visible summary counts non-empty commit lines and includes first 15 oneline entries under `subjects:`. Non-oneline forms such as `git log -5` and `git log --pretty=oneline` pass through unchanged.
### `cargo test tool_output_compression`
Summary keeps failing test names and first panic/failure lines so provider can continue debugging without receiving entire test logs.
## Developer extension process
Implementation lives at provider-visible boundary:
| File | Role |
| --- | --- |
| `src/tools/mod.rs` | Defines `ToolOutputCompressionSettings`, stores settings in `ToolRuntime`, exposes crate-private getter. |
| `src/agent/tool_lifecycle.rs` | Records raw `ToolResult` to session, then builds provider output with compression setting before `ProviderToolResult`. |
| `src/agent/tool_output_compression.rs` | Pure classifier and summarizers for curated bash rules. |
| `config/example.settings.json` | Shows default-disabled setting. |
To add rule:
1. Add conservative classifier branch in `classify_bash_command`.
2. Reject compound shell operators unless rule explicitly handles them safely.
3. Summarize only existing `ToolResult.metadata` / `ToolResult.content`; never re-run command.
4. Keep output deterministic: stable field order, fixed caps, no timestamps unless already in raw output and useful.
5. Add unit tests for disabled passthrough, non-match passthrough, matched summary, and compound-command passthrough.
6. Update this doc and [Tools and safety model](tools-and-safety.md) if provider-visible behavior changes.
## Limitations
- No arbitrary shell-output compression.
- No compression for `read`, `ffgrep`, `hash_edit`, `write`, `parallel_subagents`, `web_search`, or `code_search`.
- No token-ratio guarantee; summaries reduce common noisy outputs but preserve correctness by passing through when unrecognized.
- No shell parsing beyond conservative token/operator checks. Exact fd redirects `2>&1` / `1>&2` and safe filter pipes are the only supported compound shell forms.
- Safe pipe whitelist is fixed: `grep`, `rg`, `tail`, `head`, `sed`, `cat`, `wc`, `sort`, `uniq`, `cut`, `tr`. Other targets pass through unchanged.
- No per-rule settings or user-editable caps.
- Raw output can still be large in local display/session artifacts because preservation is intentional.
## Related docs
- [Configuration](configuration.md)
- [Tools and safety model](tools-and-safety.md)
- [Sessions, context, and cache](sessions-context-cache.md)
---
[Back to feature docs](README.md) · [Back to repository README](../../README.md)