Expand description
Safe Rust bindings to the LiteRT-LM C API 1.0.0, covering every public header of Google’s official prebuilt package:
| Header | Rust API |
|---|---|
engine.h | EngineSettings, Engine, Session, sampler / decoding configs, tokenizer, streaming, benchmarking |
conversation.h | Conversation, ConversationConfig, ConversationOptionalArgs |
embedding_engine.h | EmbeddingEngine, EmbeddingEngineSettings, EmbeddingOptions |
model_info.h | LoadedFile |
experimental.h | experimental (no stability guarantees) |
error_reporter.h | Error, StatusCode, last_error_message |
§What changed in 1.0.0
The C API is now fully status-code based, so nearly every operation
returns Result. Highlights:
Errorcarries the C status code and the library’s thread-local error message, captured at the moment of failure.- Sessions and conversations keep their
Enginealive, so the C API’s “destroy children before the engine” rule cannot be violated. - After
cancel_process, a session/conversation can only be dropped; this is enforced (Error::UseAfterCancel). - The loaded library’s C API version is verified once before the first
engine is created (
check_api_version). - New:
SpeculativeDecodingConfig,Session::clone_session, single-threaded execution,EmbeddingEngine,LoadedFile,experimental.
Setters that can only fail on a NULL pointer (impossible here) keep the
chainable &mut Self style; setters that can really fail return
Result<&mut Self>.
See the README for a function-by-function list.
Re-exports§
pub use stream::StreamChunk;pub use litertlm_sys;
Modules§
- experimental
- Experimental APIs (
experimental.h). - stream
- Streaming support shared by
ConversationandSession.
Structs§
- ApiVersion
- A C API version (
MAJOR.MINOR.PATCH). - Benchmark
Info - Benchmark data. Requires
EngineSettings::enable_benchmarkbefore the engine is created. WrapsLiteRtLmBenchmarkInfo. - Conversation
- A chat-style conversation with message history, tools, thinking, etc.
- Conversation
Config - Wraps
LiteRtLmConversationConfig. - Conversation
Optional Args - Per-turn options for
send_message*. WrapsLiteRtLmConversationOptionalArgs. - Detokenize
Result - Wraps
LiteRtLmDetokenizeResult. - Embedding
Engine - An embedding engine. Wraps
LiteRtLmEmbeddingEngine. NotSync: calls on one engine must not overlap. - Embedding
Engine Settings - Settings for an
EmbeddingEngine. WrapsLiteRtLmEmbeddingEngineSettings. - Embedding
Options - Per-request embedding options. Wraps
LiteRtLmEmbeddingOptions. - Embedding
Response - One embedding vector. Wraps
LiteRtLmEmbeddingResponse. - Embedding
Response Ref - An embedding vector borrowed from an
EmbeddingResponsesbatch. - Embedding
Responses - A batch of embedding vectors. Wraps
LiteRtLmEmbeddingResponses. - Engine
- A LiteRT-LM engine: the loaded model plus shared runtime resources.
- Engine
Settings - Builder for
LiteRtLmEngineSettings. - Input
Data - One piece of model input (text, image or audio bytes, or an
image/audio end marker). Wraps
LiteRtLmInputData. - Loaded
File - A
.litertlmfile loaded for capability queries. WrapsLiteRtLmLoadedFile. - NoRepeat
Ngram Config - Wraps
LiteRtLmNoRepeatNgramConfig. - Repetition
Penalty Config - Wraps
LiteRtLmRepetitionPenaltyConfig. - Responses
- Candidate responses from a session. Wraps
LiteRtLmResponses. - Sampler
Params - Sampling parameters. Wraps
LiteRtLmSamplerParams. - Session
- A low-level session: explicit prefill/decode control, checkpoints,
scoring and streaming. Prefer
Conversationfor chat-style use. - Session
Config - Wraps
LiteRtLmSessionConfig. - Speculative
Decoding Config - Wraps
LiteRtLmSpeculativeDecodingConfig. Attach it to anEngineSettingsand/or aSessionConfig. - Suppress
Tokens Config - Wraps
LiteRtLmSuppressTokensConfig. - Thinking
Config - Wraps
LiteRtLmThinkingConfig. - Token
Union - A token union owned by the caller, such as the one returned by
Engine::get_start_token. WrapsLiteRtLmTokenUnion; released withlitert_lm_token_union_delete. - Token
Union Ref - A token union borrowed from a
TokenUnionscollection. It is owned by the collection and must not be (and cannot be) deleted separately; the borrow checker ties it to the collection’s lifetime. - Token
Unions - A collection of token unions, e.g. an engine’s stop tokens. Wraps
LiteRtLmTokenUnions. - Tokenize
Result - Wraps
LiteRtLmTokenizeResult.
Enums§
- Activation
Data Type - Wraps
LiteRtLmActivationDataType. - Backend
Type - Wraps
LiteRtLmBackendType. - Constraint
Provider Type - Wraps
LiteRtLmConstraintProviderType. “No provider” is expressed in Rust asOption::None(seeConversationConfig::set_constraint_provider). - Constraint
Type - Wraps
LiteRtLmConstraintType, for constrained decoding. - Error
- Errors returned by this crate.
- Input
Data Type - Wraps
LiteRtLmInputDataType. - Input
Overflow Strategy - Wraps
LiteRtLmInputOverflowStrategy: what the embedding engine does when the input is longer than the model’s maximum input length. - LogSeverity
- Wraps
LiteRtLmLogSeverity. - Modality
- Wraps
LiteRtLmModality. - Model
Type - Wraps
LiteRtLmModelType. - NpuBrand
- Wraps
LiteRtLmNpuBrand. - Sampler
Type - Wraps
LiteRtLmSamplerType. - Speculative
Decoding Mode - Wraps
LiteRtLmSpeculativeDecodingMode(e.g. multi-token prediction). The modes match the--speculative_decodingflag of the LiteRT-LM CLI. - Status
Code - Mirror of
LiteRtLmStatusCode(the canonical Google/absl status codes), minusOk. - Token
Union Type - Wraps
LiteRtLmTokenUnionType.
Functions§
- api_
version - Queries the C API version of the loaded library
(
litert_lm_get_api_version). - check_
api_ version - Verifies that the loaded library is compatible with the headers this crate was built against. The (cheap) library query is cached after the first success.
- clear_
last_ error - Clears the calling thread’s last-error state
(
litert_lm_clear_last_error). Optional; seeerror_reporter.h. - extract_
text - Extracts and concatenates just the plain text from a message JSON object
of the shape
{"role": "...", "content": [{"type": "text", "text": "..."}, ...]}– i.e. whatConversation::send_messageor each stream chunk’sStreamChunk::textreturns. Non-text content blocks are skipped. Returns an empty string if the JSON doesn’t match the expected shape, since callers usually just want “whatever text is in here, if any”. - last_
error_ message - Returns the last error message recorded by the C library on the calling thread, if any.
- set_
min_ log_ level - Sets the minimum log severity for the underlying C++ library. Call this
once, before creating an
Engine, to quiet (or increase) its stderr logging. It writes straight to stderr, independent of anything in Rust, so this is the only way to control it.
Type Aliases§
- Result
- Convenience alias for
Result<T, litertlm_rs::Error>.