Crate datacortex_core 

DataCortex – lossless JSON/NDJSON compression engine.

Combines format-aware preprocessing (schema inference, columnar reorg, typed encoding) with bit-level context mixing and entropy coding to achieve compression ratios that beat zstd-19 and brotli-11 on every JSON file tested.

Quick Start

use datacortex_core::codec::{compress_to_vec, decompress_from_slice};
use datacortex_core::dcx::Mode;

let data = br#"{"id":1,"name":"test"}"#;
let compressed = compress_to_vec(data, Mode::Fast, None).unwrap();
let original = decompress_from_slice(&compressed).unwrap();
assert_eq!(data.as_slice(), original.as_slice());

Re-exports

pub use codec::compress;

pub use codec::compress_turbo;

pub use codec::compress_with_model;

pub use codec::compress_with_options;

pub use codec::decompress;

pub use codec::decompress_with_model;

pub use codec::raw_zstd_compress;

pub use codec::read_header;

pub use dcx::DcxHeader;

pub use dcx::FormatHint;

pub use dcx::Mode;

pub use format::detect_format;

pub use model::CMConfig;

pub use model::CMEngine;

Modules

codec

Codec orchestrator — compress and decompress through the DataCortex pipeline.

dcx

.dcx file format — v3 header with CRC-32 integrity.

entropy

Entropy coding — binary arithmetic coder (12-bit precision, carry-free).

format

Format detection and preprocessing pipeline.

mixer

Mixer — logistic transforms, dual logistic mixer, hierarchical mixer, APM cascade, and ISSE chain.

model

Context models — predict next bit probability from context.

state

State primitives — StateTable, StateMap, ContextMap for bit-level state tracking.