Struct SemanticCompressor 

pub struct SemanticCompressor { /* private fields */ }

Semantic-aware JSON response compressor.

Accepts a user task context string (e.g. "今天天气怎么样") and compresses the JSON response by dropping or truncating fields based on their relevance.

Implementations

impl SemanticCompressor

pub fn new() -> Self

Create a new compressor with default settings.

Level 2 (ONNX) is NOT loaded — call [load_onnx] to enable it.

pub fn load_onnx(&mut self) -> Result<bool, EmbedderError>

Attempt to load the ONNX embedding model.

Downloads model files on first use if they are not cached in ~/.tokenfleet-ai/tokenless/models/. Returns Ok(true) if Level 2 is ready, Ok(false) if the model is unavailable (Level 1 fallback).

When the onnx feature is disabled at compile time, always returns Ok(false) (Level 1 only).

pub fn compress(&self, value: &Value, context: &str) -> Value

Compress a JSON value using semantic rules.

With the onnx feature and a loaded embedder: uses cosine similarity between field names and the task context. Fields below threshold are dropped.

Without ONNX: uses keyword-based TOML rules (Level 1).

pub fn is_field_kept(&self, field_name: &str, context: &str) -> bool

Check whether a field name should be kept regardless of truncation limits, based on the user’s task context.

pub fn detect_category(&self, context: &str) -> &'static str

Return the context category detected from the given text.

Trait Implementations

impl Debug for SemanticCompressor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SemanticCompressor

fn default() -> Self

Returns the “default value” for a type.