Skip to main content

Crate tokenless_semantic

Crate tokenless_semantic 

Source
Expand description

Semantic-aware JSON field compression.

§Levels

Level 1 (default, zero deps): keyword-based context detection with compiled-in TOML profiles. No model download required.

Level 2 (onnx feature): ONNX embedding model (all-MiniLM-L6-v2, ~15 MB) that computes cosine similarity between field names and the user’s task context. Model files are auto-downloaded on first use from GitHub Releases and cached in ~/.tokenless/models/. Falls back to Level 1 automatically when the model is unavailable.

Structs§

SemanticCompressor
Semantic-aware JSON response compressor.

Enums§

EmbedderError
Errors that can occur during semantic compression.