Crate tokenfold_core 

Re-exports

pub use budget::CacheBoundary;

pub use budget::CompressionMode;

pub use budget::CompressionPolicy;

pub use budget::LossyPath;

pub use budget::TaskScope;

pub use errors::TokenFoldError;

pub use input::CompressionInput;

pub use input::CompressionOutput;

pub use input::InputFormat;

pub use modes::TransformId;

pub use pipeline::compress;

pub use pipeline::compress_with_estimator;

pub use status::Status;

Modules

budget

errors

filters

F-047: declarative command-output filter registry (roadmap.md F-047, interfaces.md §7.2 “Filter Pack Contract” and §7.3 “Filter Trust Contract”).

input

modes

Canonical mode matrix: the single source of truth for which transforms run in which mode, at what ratio cap, for which task scopes and input formats. secret_redaction is deliberately absent from this table — it runs unconditionally before the pipeline and cannot be disabled (see budget::CompressionPolicyBuilder::build).

pipeline

report

retrieval_store

F-045: reversible evidence store and retrieval (roadmap.md F-045, interfaces.md “Retrieval Marker Grammar” and the [retrieval] tokenfold.toml schema block).

safety

Fail-closed safety invariants (INTERFACES.md Part 2 “Safety Invariants on Order”). The pipeline checks these after every transform; a violation rolls that transform back (pre-transform bytes restored) rather than shipping a corrupted or unsafe output.

stats

F-046: savings ledger, stats aggregation, and JSON/CSV export (roadmap.md F-046, interfaces.md §7.1 “Stats and Analytics JSON”).

status

token_estimator

transforms

Transform implementations. Each submodule exposes a pure function over bytes/text; the pipeline (crate::pipeline) owns all bookkeeping (token counts, TransformReport, safety validation/rollback). Canonical IDs and ordering live in crate::modes.