klava-compress 0.1.4

Context compression extension for klava: drop low-value tokens from tool output with kompress.
klava-compress-0.1.4 is not a library.

klava-compress

Context compression for klava.

Large tool results are mostly boilerplate: repeated log lines, JSON scaffolding, stack frames nobody reads. This standalone extension scores every token with the kompress-v2-base model and drops low-value words, shrinking what is sent to the model.

klava launches this binary as a child process and speaks JSON over stdio; it is never compiled into the klava binary. Installing needs no rebuild of klava.

Install

cargo install klava-compress

# Install it; klava asks the binary for its [setup] hook and fetches the model:
klava extensions install ~/.cargo/bin/klava-compress --trust

The --trust above also consents to the model download, which runs right after the extension is enabled (see Model). To install and enable without downloading anything, or to fetch later:

klava extensions install ~/.cargo/bin/klava-compress --trust --no-setup
klava extensions setup klava-compress

--no-setup leaves the extension pending; it fails open until the model arrives. The same hook can be run by hand:

klava-compress --fetch-model              # into the cache
klava-compress --fetch-model --dir /models

The download resumes a partial .part file, so an interrupted 573 MB fetch does not start over.

Model

The model lands in $XDG_CACHE_HOME/klava/kompress (default ~/.cache/klava/kompress) and is never committed. The directory is resolved in this order: --dir, model_dir (from your klava settings, passed to the hook in $KLAVA_EXTENSION_CONFIG), $KLAVA_COMPRESS_MODEL_DIR, then the cache. scripts/fetch-model.sh does the same with curl and remains for manual use.

Configuration

All keys live under [extensions.settings.klava-compress] in the klava config. klava extensions show klava-compress prints them with their defaults:

[extensions.settings.klava-compress]
enabled = true
# model_dir = "~/.cache/klava/kompress"
# threshold = 0.7
# min_tokens = 200
# max_tokens = 1500
# min_savings = 0.10
# roles = ["tool"]
# skip_structured = true
Key Default Meaning
enabled false Master switch. The model is a ~580 MB download, so this is an explicit choice.
model_dir cache dir Where kompress-fp32.onnx and tokenizer.json live. Empty = $KLAVA_COMPRESS_MODEL_DIR, else $XDG_CACHE_HOME/klava/kompress.
threshold 0.7 Keep a word when any of its tokens scores ≥ this. Lower keeps more (and saves less).
min_tokens 200 Skip shorter messages; the fixed inference cost dominates.
max_tokens 1500 Skip longer messages; inference is superlinear (a 3.4k-token message costs ~6.5 s).
min_savings 0.10 Discard a result unless it removes at least this fraction of words.
roles ["tool"] Which message roles may be rewritten. Never touches system or assistant instructions.
skip_structured true Skip content that is mostly code, SQL, JSON, or diffs.

Safety

  • Removal is word-level, never token-level, so identifiers (EIO-7734), numbers and whitespace structure survive.
  • Every rewrite is checked against the original and discarded if it would drop a negation, a number, an identifier, an ALL-CAPS marker, or a severity word.
  • Fails open: missing model, bad tokenizer, inference error, structured content, or an unsafe rewrite all leave the request unchanged.

See klava's EXTENSIONS.md for the wire protocol.