Crate marqant

Marqant - Quantum-Compressed Markdown Format

Marqant (.mq) is a revolutionary compression format designed specifically for AI consumption, achieving 90% token reduction while maintaining semantic integrity.

Features

• Token-based compression: Common markdown patterns become single tokens
• AI-optimized: Reduces token usage in LLM contexts by 70-90%
• Streaming support: Can process before full dictionary is loaded
• Multiple compression levels: From light tokenization to quantum compression
• DNS integration: Supports distributed token dictionaries via DNS

Usage

use marqant::Marqant;

let compressor = Marqant::default();
let compressed = compressor.compress("# Hello World\n\nThis is markdown content");
let decompressed = compressor.decompress(&compressed).unwrap();

Binary Format

The .mq format consists of:

• Header: Version, timestamp, sizes, flags
• Token dictionary with escaped patterns
• Compressed content using token substitution
• Optional metadata sections

Modules

dns

novelty

semantic

Structs

Marqant

The main Marqant compressor

MqInfo

Constants

MQ2_UNI_DICT_ID

Functions

mq2_uni_decode

mq2_uni_encode

read_mq_metadata

wordcloud_svg

Generate a deterministic SVG word cloud from input text.