Crate llguidance 

Constrained decoding (structured outputs) for Large Language Models.

This crate enforces arbitrary context-free grammars on LLM output, enabling structured generation with negligible overhead (~50μs per token for a 128k tokenizer). Context-free grammars can be provided with a Lark-like syntax, with specialised support for JSON.

Key types

• ParserFactory — compiles grammars and holds shared tokenizer state.
• TokenParser — built via ParserFactory::create_parser(); drives a single generation session.
• Constraint — main entry point wrapping a TokenParser and exposing the sampling-loop API.

Usage pattern

1. Call Constraint::compute_mask() to obtain the set of allowed tokens. This may take >1 ms and is best run on a background thread.
2. Sample a token from the LLM using the mask.
3. Pass the sampled token to Constraint::commit_token() (very fast).
4. Repeat until a stop result is returned.

See the sample_parser crate for a complete usage example.

Re-exports

pub use toktrie;

pub use derivre;

Modules

api

earley

This is the primary interface for llguidance – the one on which the others (FFI and LLInterpreter) are built. While not cleanest of these interfaces, it is the most inclusive.

ffi

output

panic_utils

substring

Macros

id32_type

infoln

loginfo

warn

Structs

CommitResult

Constraint

High-level entry point for constrained decoding.

GrammarBuilder

Instant

A measurement of a monotonically nondecreasing clock. Opaque and useful only with Duration.

JsonCompileOptions

Logger

Matcher

This is meant to be used in server-side scenarios. The Constraint interface is more for usage in Python Guidance.

NodeRef

ParserFactory

Compiles grammars and holds shared tokenizer state.

StopController

TokenParser

Token-level parser that drives a single constrained-generation session.

Functions

json_merge

regex_to_lark

Make sure given regex can be used inside /…/ in Lark syntax. Also if use_ascii.contains('d') replace \d with [0-9] and \D with [^0-9]. Similarly for \w/\W ([0-9a-zA-Z_]) and \s/\S ([ \t\n\r\f\v]). For standard Unicode Python3 or Rust regex crate semantics use_ascii = "" For JavaScript or JSON Schema semantics use_ascii = "dw" For Python2 or byte patters in Python3 semantics use_ascii = "dws" More flags may be added in future.

token_bytes_from_tokenizer_json

Parse HF tokenizer.json file and return bytes for every token

Type Aliases

HashMap

HashSet