Struct GrammarCompiler 

pub struct GrammarCompiler { /* private fields */ }

The compiler for grammars. It is associated with a certain tokenizer info, and compiles grammars into CompiledGrammar with the tokenizer info. It allows parallel compilation with multiple threads, and has a cache to store the compilation result, avoiding compiling the same grammar multiple times.

Implementations

impl GrammarCompiler

pub fn new( tokenizer_info: &TokenizerInfo, max_threads: i32, cache_enabled: bool, cache_limit_bytes: isize, ) -> Self

Construct the compiler.

Parameters

• tokenizer_info: The tokenizer info.
• max_threads (default: 8): The maximum number of threads used to compile the grammar.
• cache_enabled (default: true): Whether to enable the cache.
• cache_limit_bytes (default: -1): The maximum memory usage for the cache in bytes. Note that the actual memory usage may slightly exceed this value.

pub fn compile_json_schema( &mut self, schema: &str, any_whitespace: bool, indent: Option<i32>, separators: Option<(impl AsRef<str>, impl AsRef<str>)>, strict_mode: bool, max_whitespace_cnt: Option<i32>, ) -> CompiledGrammar

Get CompiledGrammar from the specified JSON schema and format. The indent and separators parameters follow the same convention as in serde_json’s pretty printing (mirroring Python’s json.dumps()).

Parameters

• schema: The schema string.
• any_whitespace: Whether to allow any whitespace regardless of indent/separators.
• indent: The number of spaces for indentation. If None, the output will be in one line.
• separators: Two separators used in the schema: comma and colon. Examples: (“,”, “:”), (“, “, “: “). If None, defaults to (”,“, “: “) when indent is Some, otherwise (”, “, “: “).
• strict_mode: Whether to use strict mode. In strict mode, the generated grammar will not allow properties and items that are not specified in the schema. This is equivalent to setting unevaluatedProperties and unevaluatedItems to false.

pub fn compile_builtin_json_grammar(&mut self) -> CompiledGrammar

Get CompiledGrammar from the standard JSON.

pub fn compile_regex(&mut self, regex: &str) -> CompiledGrammar

Get CompiledGrammar from the specified regex.

pub fn compile_structural_tag( &mut self, tags: &[StructuralTagItem], triggers: &[impl AsRef<str>], ) -> CompiledGrammar

Compile a grammar from structural tags.

Parameters

• tags: The structural tags.
• triggers: The triggers. Each trigger should be a prefix of a provided begin tag.

pub fn compile_grammar(&mut self, grammar: &Grammar) -> CompiledGrammar

Compile a grammar object to a CompiledGrammar.

pub fn compile_grammar_from_ebnf( &mut self, ebnf_string: &str, root_rule_name: &str, ) -> CompiledGrammar

Compile a grammar from an EBNF string. The string should follow the format described in https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md

Parameters

• ebnf_string: The grammar string in EBNF format.
• root_rule_name: The name of the root rule in the grammar.

pub fn clear_cache(&mut self)

Clear all cached compiled grammars.

pub fn get_cache_size_bytes(&self) -> i64

The approximate memory usage of the cache in bytes.

pub fn cache_limit_bytes(&self) -> i64

The maximum memory usage for the cache in bytes. Returns -1 if unlimited.

Trait Implementations

impl Drop for GrammarCompiler

fn drop(&mut self)

Executes the destructor for this type.