Crate pithy_core 

pithy-core - Encoder decoder measurer format selector tokenizer registry fallback cascade session symbol table

Licensed under PolyForm Noncommercial 1.0 + Pithy Attribution Rider. Commercial use requires a separate license. See COMMERCIAL.md.

Copyright (c) 2026 Mikko Parkkola. All rights reserved.

Re-exports

pub use branding::ATTRIBUTION;

pub use dialect::pick_dialect;

pub use dialect::rules_for;

pub use dialect::Dialect;

pub use dialect::Rule;

pub use encoder::encode_symbolic;

pub use encoder::encode_symbolic_traced;

pub use encoder::encode_symbolic_traced_with;

pub use encoder::EncoderTrace;

pub use encoder::RuleSet;

pub use encoder::SymbolicEncoder;

pub use encoder::ENABLE_WEIGHT_THRESHOLD;

pub use encoder::MAX_INPUT_CHARS;

pub use encoder::MIN_INPUT_CHARS;

pub use encoder::RULE_NAMES;

pub use interfaces::Compressed;

pub use interfaces::CostEntry;

pub use interfaces::Encoder;

pub use interfaces::FallbackReason;

pub use interfaces::Format;

pub use interfaces::Integrity;

pub use interfaces::LogError;

pub use interfaces::MeasurementLog;

pub use interfaces::Measurer;

pub use interfaces::Model;

pub use interfaces::QualityScore;

pub use interfaces::Record;

pub use interfaces::RecordId;

pub use interfaces::Signer;

pub use interfaces::SignerError;

pub use interfaces::StreamDelta;

pub use interfaces::TokenizerError;

pub use session::Binding;

pub use session::SessionPolicy;

pub use session::SessionSymbolTable;

pub use tokenizers::LocalMeasurer;

pub use tokenizers::TokenizerId;

pub use tokenizers::TokenizerRegistry;

Modules

branding

Single source of truth for product branding.

dialect

Dialect dispatcher - picks grammar rule set per tokenizer generation.

encoder

Symbolic encoder – Rust port of research/f2_selector_oracle.py::encode_symbolic.

interfaces

Pithy interface contracts — FROZEN at Phase 0.

session

Session-scoped symbol table – patent claim 6.

tokenizers

Tokenizer registry and Measurer implementation.

Constants

VERSION

Crate semantic version (read from Cargo.toml at compile time).