ferrox_models/grammar/
mod.rs

1//! GBNF grammar-constrained decoding.
2//!
3//! A port of llama.cpp's `src/llama-grammar.cpp` and `src/llama-grammar.h`
4//! (MIT; see `docs/THIRD_PARTY_NOTICES.md`). The algorithm is transcribed
5//! from that source rather than reconstructed from how EBNF engines
6//! usually work, because the two differ in places that matter -- the
7//! negation of a character class lives on its first element only, the
8//! repetition rewrites produce observable rule ids, and a token piece that
9//! ends mid-codepoint has to stay viable rather than be rejected.
10//!
11//! What this replaces, and why it is not the same kind of thing:
12//! `crates/ferrox-server/src/json_mode.rs` masks logits by a fixed
13//! character class. It has no state, so it cannot know whether a `}`
14//! closes an object that was opened. This is a pushdown machine over a
15//! parsed grammar, so it can.
16//!
17//! # Layout
18//!
19//! | Module | Role |
20//! |---|---|
21//! | [`element`] | the compiled element / rule / cursor types |
22//! | [`error`] | every refusal, naming what is missing |
23//! | [`utf8`] | llama.cpp's two UTF-8 decoders, partial sequences included |
24//! | [`parser`] | GBNF text to a rule table |
25//! | [`machine`] | the pushdown stack machine over that table |
26//! | [`candidates`] | which candidate tokens no viable stack accepts |
27//! | [`lazy`] | trigger tokens / patterns: a grammar that switches on mid-generation |
28//! | [`json_schema`] | JSON Schema to GBNF text, for `response_format` |
29//!
30//! # Status
31//!
32//! Steps 1 and 2 of the three-step spine in
33//! `docs/plans/llama-cpp-gap-inventory.md` §8 item 8: the parser and the
34//! stack machine, with the candidate-rejection core the sampler hook would
35//! sit on. **Not wired into sampling or the server.** The seam a caller
36//! needs is [`machine::Grammar::accept_token`] after each accepted token
37//! and [`candidates::reject_candidates`] before each sample; nothing calls
38//! either yet.
39//!
40//! [`json_schema`] now ports `common/json-schema-to-grammar.cpp`, which is
41//! what `response_format: json_schema` needs on top of a grammar. It is
42//! stricter than upstream by design -- a keyword it cannot honour is a
43//! typed refusal rather than a grammar that accepts too much -- and its
44//! module docs list what is ported, what is refused by name, and where its
45//! output differs from llama.cpp's. `ferrox-server` now answers
46//! `response_format: {"type": "json_schema"}` through it (see that crate's
47//! `grammar_request`), so a schema and a hand-written grammar reach the
48//! same machine.
49//!
50//! **Lazy grammars** ([`lazy`]) are ported: `trigger_tokens` and
51//! `trigger_patterns`, the accumulated trigger buffer, and the replay that
52//! feeds the grammar from the match onward. That is what `tool_choice`
53//! needs on top of a grammar -- a model may talk before it calls a tool,
54//! and a grammar applied from token zero forbids the talking.
55
56pub mod candidates;
57pub mod element;
58pub mod error;
59pub mod json_schema;
60pub mod lazy;
61pub mod machine;
62pub mod parser;
63pub mod utf8;
64
65#[cfg(test)]
66mod grammars;
67#[cfg(test)]
68mod machine_tests;
69#[cfg(test)]
70mod parser_tests;
71
72pub use candidates::{reject_candidates, Candidate, DecodedPiece};
73pub use element::{GrammarElement, GrammarRule, GrammarStack, GreType, RulePos};
74pub use error::GrammarError;
75pub use json_schema::{json_schema_to_grammar, json_schema_to_grammar_value, SchemaError};
76pub use lazy::{LazyState, LazyTriggers, TriggerPattern, TriggerStep};
77pub use machine::Grammar;
78pub use parser::{parse, parse_with_vocab, GrammarVocab, ParsedGrammar};
79pub use utf8::PartialUtf8;
ferrox_models/grammar/mod.rs

ferrox_models/grammar/
mod.rs
