Crate baseRT_sys 

Raw FFI bindings for the BaseRT C API.

These are unsafe, low-level bindings. Prefer the baseRT crate for safe usage.

Structs

BaseRTGenerationStats

Generation result statistics.

BaseRTModelConfig

Model configuration extracted from weight file metadata.

BaseRTPrefixMatch

Result of a prefix-cache lookup (baseRT_prefix_match). blocks points into engine-owned storage valid until baseRT_prefix_unlock(handle) / baseRT_prefix_release(handle). handle == 0 means miss / cache disabled — nothing to unlock.

BaseRTSamplingConfig

Sampling configuration for text generation.

BaseRTTranscribeSegment

One segment of the last transcription (verbose_json surface). text is borrowed from the engine — valid until the next transcription or model free.

BaseRTTranscribeStats

Transcription result statistics.

Functions

baseRT_abi_version^⚠

The library’s BASERT_ABI_VERSION (since 0.3.1).

baseRT_argmax_logits_row^⚠

baseRT_batch_logits_stride^⚠

baseRT_batch_step^⚠

baseRT_batch_step_fused^⚠

baseRT_batch_step_fused_logits^⚠

baseRT_batch_warmup^⚠

baseRT_bos_id^⚠

baseRT_capabilities^⚠

baseRT_chain_decode^⚠

baseRT_chat_template^⚠

baseRT_chat_template_jinja^⚠

Raw Jinja chat_template source folded in from the HF tokenizer (empty if the bundle carries none).

baseRT_context_window_fits^⚠

1 = the window fits the co-resident set, 0 = it does not, -1 = unknown.

baseRT_decode_step^⚠

baseRT_decode_token^⚠

baseRT_decode_token_raw^⚠

Length-preserving stateless decode (byte-level tokens can contain NUL); returns the true byte length, which may exceed max_bytes.

baseRT_device_memory_budget^⚠

Device memory budget in bytes (Metal: recommended working set; CUDA: total device memory). 0 when no supported device is present.

baseRT_embed^⚠

baseRT_embed_text^⚠

baseRT_embedding_dim^⚠

baseRT_encode^⚠

baseRT_encode_pieces^⚠

n_pieces texts encoded as one: piece i parses marker strings when plain[i] is zero and keeps them as text otherwise, with the pre-tokenizer seeing the concatenation. See the header.

baseRT_encode_plain^⚠

Special-token strings in text are ordinary text (client content), never control ids. See the header.

baseRT_eos_token_id^⚠

baseRT_format_chat^⚠

baseRT_free_model^⚠

baseRT_gdn_snapshot_size^⚠

baseRT_generate^⚠

baseRT_generate_continue^⚠

baseRT_get_config^⚠

baseRT_get_config_into^⚠

The configuration copied into a caller-sized buffer, zero-filled past the library’s struct; returns the library’s sizeof (since 0.4.0).

baseRT_get_error^⚠

baseRT_get_error_code^⚠

baseRT_get_position^⚠

baseRT_gpu_repetition_penalty^⚠

baseRT_gpu_temperature_scale^⚠

baseRT_grammar_accept_token^⚠

Step a grammar by one token; 0 = the grammar REJECTS it. Lets a test assert what a compiled grammar actually admits, not merely that it compiled — the difference between “the tag parses” and “the tag permits the calls this dialect must be able to make”.

baseRT_grammar_create^⚠

Create a grammar from a GBNF string; NULL on parse error.

baseRT_grammar_create_from_schema^⚠

Create a grammar from a JSON Schema (converted to GBNF internally); NULL on error. Used to validate that a codec’s tool-call grammar compiles in the engine (the runtime constrained-decode path).

baseRT_grammar_create_from_structural_tag^⚠

Create a grammar from an xgrammar STRUCTURAL TAG document (JSON, not a grammar string); NULL on error. The tag’s content formats reach past JSON Schema — sequence, const_string, regex, or — which is how a dialect whose call body is not JSON (Gemma 4’s call:fn{k:v}) still gets an enforceable tool_choice.

baseRT_grammar_free^⚠

Free a grammar.

baseRT_grammar_reset^⚠

Reset a grammar’s acceptance state to its initial stacks, so one handle can check several candidate strings.

baseRT_is_embedding_model^⚠

baseRT_is_eos_token^⚠

baseRT_is_whisper^⚠

baseRT_kv_bits_effective^⚠

baseRT_load_model^⚠

baseRT_load_tokenizer_only^⚠

Tokenizer-only load: metadata + vocab, no GPU tensors. Only the tokenizer surface is valid on the returned handle.

baseRT_logits_row_logprobs^⚠

baseRT_mask_logits_row^⚠

baseRT_max_prefill_chunk^⚠

baseRT_model_config_sizeof^⚠

baseRT_model_memory^⚠

baseRT_page_size^⚠

baseRT_prefill^⚠

baseRT_prefix_cache_stats^⚠

baseRT_prefix_discard^⚠

baseRT_prefix_evict^⚠

baseRT_prefix_insert^⚠

baseRT_prefix_match^⚠

baseRT_prefix_release^⚠

baseRT_prefix_unlock^⚠

baseRT_profile_decode_step^⚠

baseRT_profile_label^⚠

baseRT_read_batch_logits^⚠

baseRT_reset^⚠

baseRT_sample_logits_row^⚠

baseRT_sequence_create^⚠

baseRT_sequence_free^⚠

baseRT_sequence_gdn_capture^⚠

baseRT_sequence_gdn_restore^⚠

baseRT_sequence_rollback^⚠

baseRT_sequence_seed_prefix^⚠

baseRT_set_baked_decode^⚠

baseRT_set_condition_on_previous_text^⚠

baseRT_set_initial_prompt^⚠

baseRT_set_kv_bits^⚠

baseRT_set_max_batch_size^⚠

baseRT_set_paged_kv^⚠

baseRT_set_prefill_chunk^⚠

baseRT_set_prefix_cache^⚠

baseRT_set_speculation^⚠

baseRT_set_task^⚠

baseRT_set_timestamps^⚠

baseRT_set_verbose^⚠

baseRT_special_token^⚠

The index-th special token string; writes its id to id_out.

baseRT_special_token_count^⚠

Number of added/special tokens the model declares.

baseRT_strerror^⚠

baseRT_suggest_max_context^⚠

A max_context sized for this device and this bundle. Metadata-only, so it is safe to call before loading. 0 = could not size.

baseRT_suggest_max_context_multi^⚠

As above for a set of models that will be resident at the same time: weights and KV pools add up, and the trained-window cap is the shortest among them.

baseRT_suggest_max_context_spec^⚠

As above, plus the speculators loading beside those models: speculator_embedded[i] non-zero is the target bundle’s embedded head (its KV pool only); zero (or a null array) a sidecar / draft model (weights and KV pool), even when also served. speculator_target[i]

baseRT_tensor_count^⚠

baseRT_tensor_dtype^⚠

baseRT_tensor_name^⚠

baseRT_tensor_raw_dtype^⚠

baseRT_token_count^⚠

baseRT_transcribe^⚠

baseRT_transcribe_audio_duration_ms^⚠

baseRT_transcribe_language^⚠

baseRT_transcribe_pcm^⚠

baseRT_transcribe_pcm_stream^⚠

baseRT_transcribe_segment^⚠

baseRT_transcribe_segment_count^⚠

baseRT_transcribe_stream^⚠

baseRT_weights_identity^⚠

Type Aliases

baseRT_model_t

Opaque model handle.

baseRT_segment_callback

Callback for streaming transcription segments. Return false to stop transcription.

baseRT_sequence_t

Opaque per-sequence handle (independent KV state on the model’s shared paged-KV pool). Mirrors baseRT_sequence_t in baseRT.h.

baseRT_token_callback

Callback for streaming token output. Return false to stop generation.