Expand description
Constrained-decoding grammar processors.
Submodules:
schema— internalSchemaenum and JSON-Schema parser (subset).state—JsonGrammarstate machine that tracks where we are in the partially-emitted JSON value.json_schema— publicJsonSchemaProcessorthat wraps a tokenizer vocabulary and produces per-step token-allow masks for use withferrotorch_cubecl::apply_token_mask_to_gpu.
§REQ status (per .design/ferrotorch-grammar/lib.md)
| REQ | Status | Evidence |
|---|---|---|
| REQ-1 | SHIPPED | impl: pub mod json_schema; pub mod schema; pub mod state; + #[cfg(feature = "cuda")] pub mod gpu_dispatch; in lib.rs; non-test consumer: pub use ferrotorch_grammar as grammar; in ferrotorch-llama/src/lib.rs:156 makes the submodule tree reachable to every downstream model crate (grandfathered public API per goal.md S5). |
| REQ-2 | SHIPPED | impl: pub use json_schema::{GrammarError, JsonSchemaProcessor, TokenMask}; + pub use schema::Schema; + pub use state::{BooleanEmissionStage, JsonGrammar}; + #[cfg(feature = "cuda")] pub use gpu_dispatch::{PackedVocab, compute_mask_gpu}; in lib.rs; non-test consumer: ferrotorch-llama/src/lib.rs:156 aliases the whole crate (grandfathered S5). |
| REQ-3 | SHIPPED | impl: the re-export chain in lib.rs; non-test consumer: pub use ferrotorch_grammar as grammar; in ferrotorch-llama/src/lib.rs:156 documented at lines 150-155 is the literal alias that makes the entire public surface a member of ferrotorch_llama::grammar. |
Re-exports§
pub use json_schema::GrammarError;pub use json_schema::JsonSchemaProcessor;pub use json_schema::TokenMask;pub use json_schema::TokenTransitionCache;pub use schema::Schema;pub use state::BooleanEmissionStage;pub use state::JsonGrammar;
Modules§
- json_
schema - Public
JsonSchemaProcessor: token-level wrapper aroundJsonGrammar. - schema
Schema: a typed, internal representation of the supported JSON-Schema subset.- state
- Character-level state machine over a JSON value matching a
Schema.