Expand description
Stateful GBNF grammar + DRY samplers.
ik_llama.cpp exposes two stateful sampler objects that do not fit the
stateless llama_sample_* array API modelled in crate::sampling (they
own C-side state that advances as tokens are accepted):
LlamaGrammar— a GBNF grammar constraint. Unlike the stockllama-cpp-2anchor (which parses GBNF in Rust and calls the now commented-outllama_grammar_init), ik parses the grammar C-side viallama_sampler_init_grammar(vocab, grammar_str, root). The returned grammar retains a pointer to the model’s vocab, so it is lifetime-tied to theLlamaModelit was built from.LlamaDrySampler— the DRY (“Don’t Repeat Yourself”) repetition sampler. It processes its sequence breakers against the vocab only at init, so it does not retain the model.
Both own a raw C object and free it on Drop. Typical use inside a
generation loop: LlamaGrammar::apply / LlamaDrySampler::apply mutate
the candidate array in place before you draw a token; then
LlamaGrammar::accept_token / LlamaDrySampler::accept advance the
sampler state with the token you drew.
Structs§
- DryParams
- Parameters for the DRY (“Don’t Repeat Yourself”) sampler.
- Llama
DrySampler - The stateful DRY repetition sampler.
- Llama
Grammar - A stateful GBNF grammar constraint.
Enums§
- DryInit
Error - Error building a
LlamaDrySampler. - Grammar
Init Error - Error building a
LlamaGrammar. - Json
Schema Error - Error converting a JSON Schema into a GBNF grammar via
json_schema_to_grammar(thecommonfeature).
Functions§
- json_
schema_ to_ grammar - Convert a JSON Schema (given as a JSON string) into a GBNF grammar string,
ready to pass to
LlamaGrammar::new.