ferrotorch_grammar/lib.rs
1//! Constrained-decoding grammar processors.
2//!
3//! Submodules:
4//!
5//! - [`schema`] — internal `Schema` enum and JSON-Schema parser (subset).
6//! - [`state`] — `JsonGrammar` state machine that tracks where we are in
7//! the partially-emitted JSON value.
8//! - [`json_schema`] — public `JsonSchemaProcessor` that wraps a tokenizer
9//! vocabulary and produces per-step token-allow masks for use with
10//! `ferrotorch_cubecl::apply_token_mask_to_gpu`.
11//!
12//! ## REQ status (per `.design/ferrotorch-grammar/lib.md`)
13//!
14//! | REQ | Status | Evidence |
15//! |---|---|---|
16//! | REQ-1 | SHIPPED | impl: `pub mod json_schema; pub mod schema; pub mod state;` + `#[cfg(feature = "cuda")] pub mod gpu_dispatch;` in `lib.rs`; non-test consumer: `pub use ferrotorch_grammar as grammar;` in `ferrotorch-llama/src/lib.rs:156` makes the submodule tree reachable to every downstream model crate (grandfathered public API per goal.md S5). |
17//! | REQ-2 | SHIPPED | impl: `pub use json_schema::{GrammarError, JsonSchemaProcessor, TokenMask};` + `pub use schema::Schema;` + `pub use state::{BooleanEmissionStage, JsonGrammar};` + `#[cfg(feature = "cuda")] pub use gpu_dispatch::{PackedVocab, compute_mask_gpu};` in `lib.rs`; non-test consumer: `ferrotorch-llama/src/lib.rs:156` aliases the whole crate (grandfathered S5). |
18//! | REQ-3 | SHIPPED | impl: the re-export chain in `lib.rs`; non-test consumer: `pub use ferrotorch_grammar as grammar;` in `ferrotorch-llama/src/lib.rs:156` documented at lines 150-155 is the literal alias that makes the entire public surface a member of `ferrotorch_llama::grammar`. |
19
20pub mod json_schema;
21pub mod schema;
22pub mod state;
23
24#[cfg(feature = "cuda")]
25pub mod gpu_dispatch;
26
27pub use json_schema::{GrammarError, JsonSchemaProcessor, TokenMask, TokenTransitionCache};
28pub use schema::Schema;
29pub use state::{BooleanEmissionStage, JsonGrammar};
30
31#[cfg(feature = "cuda")]
32pub use gpu_dispatch::{PackedVocab, compute_mask_gpu};