Crate maskforge_core 

MaskForge core: bounded schema lowering, frozen IR, incremental matching, and vocabulary masks. Links without PyO3; correctness oracles are test/benchmark-only.

Architecture: https://github.com/dhanavanthesh/maskforge/blob/main/docs/architecture.md.

Re-exports

pub use api::BindPolicy;

pub use api::BoundSchema;

pub use api::CompiledVocabulary;

pub use api::Compiler;

pub use api::CompilerOptions;

pub use api::SchemaProgram;

pub use api::Session;

pub use api::SessionError;

pub use api::TrieCacheStats;

pub use automaton::RefEngine;

pub use compile::compile_ir;

pub use diagnostics::Diagnostic;

pub use diagnostics::Span;

pub use diagnostics::UnsupportedReason;

pub use error::CompileError;

pub use error::ErrorCode;

pub use error::HashState;

pub use error::LimitKind;

pub use error::MatcherError;

pub use error::Provenance;

pub use error::ProvenanceProfile;

pub use error::Stage;

pub use error::VocabError;

pub use frontend::regex_to_ir;

pub use frontend::schema_to_ir;

pub use frontend::schema_to_ir_with_external_refs;

pub use frontend::schema_to_ir_with_resources;

pub use frontend::SchemaRegistry;

pub use frontend::SchemaResourceLimits;

pub use ir::Builder;

pub use ir::Charset;

pub use ir::CompileOptions;

pub use ir::Node;

pub use ir::ObjectClosure;

pub use ir::ScalarLit;

pub use ir::SchemaIR;

pub use ir::UnsupportedPolicy;

pub use ir::IR_VERSION;

pub use mask::Bitmask;

pub use primitives::ByteClassId;

pub use primitives::MintermId;

pub use primitives::StateId;

pub use primitives::TokenId;

pub use routing::add_bound;

pub use routing::mul_bound;

pub use routing::pow2_bound;

pub use routing::Bound;

pub use routing::CostEstimate;

pub use routing::ExecutionKind;

pub use routing::NodeRoute;

pub use routing::RouteReason;

pub use routing::RouteTable;

pub use runtime::CompiledArtifact;

pub use runtime::ExecutableCache;

pub use runtime::ExecutableSchema;

pub use runtime::Matcher;

pub use structured::StructuredMatcher;

pub use structured::StructuredMatcherError;

pub use structured::StructuredProgram;

Modules

api

The stable, high-level facade: compile a schema once, prepare a vocabulary once, bind them, drive one generation sequence. Callers never inspect which backend a schema needs.

automaton

Byte-DFA wrapper and canonical-DEAD materialization. External regex-automata StateID never leaves this module; only crate::primitives::StateId does.

compile

The production compiler: a validated arena SchemaIR is lowered directly to a byte-level pattern and handed to the shared anchored-DFA builder. It consumes the typed nodes; it never re-parses JSON and never transports a regex string across the FFI.

diagnostics

The provenance sidecar. A Diagnostic records where and why an occurrence was rejected; it is keyed by occurrence and never enters the semantic hash.

error

The error spine: stable machine-readable codes are the contract; the English message is advisory.

frontend

Converts JSON Schema text into validated schema IR.

index

Binds vocabularies to automata for allowed-token mask generation.

ir

The public arena schema IR: a flat, immutable, hash-consed DAG built once through Builder and consumed directly by the compiler. All fields are private; the only ways in are Builder::finish and SchemaIR::from_wire, and the only way out is a read-only accessor.

mask

Packed-u32 allowed-token mask (the single wire contract).

primitives

The newtype spine: zero-cost, transparent ids that never appear as a bare integer in new code.

routing

Shared, bounded execution-cost analysis for regular and structured compilation.

runtime

Provides compiled artifacts and stateful matchers.

structured

Structured validation for schemas that need more than one anchored byte-DFA.

Structs

Vocabulary

Vocabulary for a large language model.

Functions

build_vocabulary

Validated Vocabulary constructor: rejects empty-byte tokens, an empty id list, and an EOS id present in the token map. Duplicate byte sequences merge; raw non-UTF-8 bytes are valid.

build_vocabulary_from_str_keys

Validated Vocabulary constructor over String keys. Same rules as build_vocabulary.

Type Aliases

StringTokenMap

The token map build_vocabulary_from_str_keys accepts, keyed by String.

TokenMap

Token bytes mapped to the ids that decode to them.