Skip to main content

Crate litertlm_rs

Crate litertlm_rs 

Source
Expand description

Safe Rust bindings to the LiteRT-LM C API 1.0.0, covering every public header of Google’s official prebuilt package:

HeaderRust API
engine.hEngineSettings, Engine, Session, sampler / decoding configs, tokenizer, streaming, benchmarking
conversation.hConversation, ConversationConfig, ConversationOptionalArgs
embedding_engine.hEmbeddingEngine, EmbeddingEngineSettings, EmbeddingOptions
model_info.hLoadedFile
experimental.hexperimental (no stability guarantees)
error_reporter.hError, StatusCode, last_error_message

§What changed in 1.0.0

The C API is now fully status-code based, so nearly every operation returns Result. Highlights:

Setters that can only fail on a NULL pointer (impossible here) keep the chainable &mut Self style; setters that can really fail return Result<&mut Self>.

See the README for a function-by-function list.

Re-exports§

pub use stream::StreamChunk;
pub use litertlm_sys;

Modules§

experimental
Experimental APIs (experimental.h).
stream
Streaming support shared by Conversation and Session.

Structs§

ApiVersion
A C API version (MAJOR.MINOR.PATCH).
BenchmarkInfo
Benchmark data. Requires EngineSettings::enable_benchmark before the engine is created. Wraps LiteRtLmBenchmarkInfo.
Conversation
A chat-style conversation with message history, tools, thinking, etc.
ConversationConfig
Wraps LiteRtLmConversationConfig.
ConversationOptionalArgs
Per-turn options for send_message*. Wraps LiteRtLmConversationOptionalArgs.
DetokenizeResult
Wraps LiteRtLmDetokenizeResult.
EmbeddingEngine
An embedding engine. Wraps LiteRtLmEmbeddingEngine. Not Sync: calls on one engine must not overlap.
EmbeddingEngineSettings
Settings for an EmbeddingEngine. Wraps LiteRtLmEmbeddingEngineSettings.
EmbeddingOptions
Per-request embedding options. Wraps LiteRtLmEmbeddingOptions.
EmbeddingResponse
One embedding vector. Wraps LiteRtLmEmbeddingResponse.
EmbeddingResponseRef
An embedding vector borrowed from an EmbeddingResponses batch.
EmbeddingResponses
A batch of embedding vectors. Wraps LiteRtLmEmbeddingResponses.
Engine
A LiteRT-LM engine: the loaded model plus shared runtime resources.
EngineSettings
Builder for LiteRtLmEngineSettings.
InputData
One piece of model input (text, image or audio bytes, or an image/audio end marker). Wraps LiteRtLmInputData.
LoadedFile
A .litertlm file loaded for capability queries. Wraps LiteRtLmLoadedFile.
NoRepeatNgramConfig
Wraps LiteRtLmNoRepeatNgramConfig.
RepetitionPenaltyConfig
Wraps LiteRtLmRepetitionPenaltyConfig.
Responses
Candidate responses from a session. Wraps LiteRtLmResponses.
SamplerParams
Sampling parameters. Wraps LiteRtLmSamplerParams.
Session
A low-level session: explicit prefill/decode control, checkpoints, scoring and streaming. Prefer Conversation for chat-style use.
SessionConfig
Wraps LiteRtLmSessionConfig.
SpeculativeDecodingConfig
Wraps LiteRtLmSpeculativeDecodingConfig. Attach it to an EngineSettings and/or a SessionConfig.
SuppressTokensConfig
Wraps LiteRtLmSuppressTokensConfig.
ThinkingConfig
Wraps LiteRtLmThinkingConfig.
TokenUnion
A token union owned by the caller, such as the one returned by Engine::get_start_token. Wraps LiteRtLmTokenUnion; released with litert_lm_token_union_delete.
TokenUnionRef
A token union borrowed from a TokenUnions collection. It is owned by the collection and must not be (and cannot be) deleted separately; the borrow checker ties it to the collection’s lifetime.
TokenUnions
A collection of token unions, e.g. an engine’s stop tokens. Wraps LiteRtLmTokenUnions.
TokenizeResult
Wraps LiteRtLmTokenizeResult.

Enums§

ActivationDataType
Wraps LiteRtLmActivationDataType.
BackendType
Wraps LiteRtLmBackendType.
ConstraintProviderType
Wraps LiteRtLmConstraintProviderType. “No provider” is expressed in Rust as Option::None (see ConversationConfig::set_constraint_provider).
ConstraintType
Wraps LiteRtLmConstraintType, for constrained decoding.
Error
Errors returned by this crate.
InputDataType
Wraps LiteRtLmInputDataType.
InputOverflowStrategy
Wraps LiteRtLmInputOverflowStrategy: what the embedding engine does when the input is longer than the model’s maximum input length.
LogSeverity
Wraps LiteRtLmLogSeverity.
Modality
Wraps LiteRtLmModality.
ModelType
Wraps LiteRtLmModelType.
NpuBrand
Wraps LiteRtLmNpuBrand.
SamplerType
Wraps LiteRtLmSamplerType.
SpeculativeDecodingMode
Wraps LiteRtLmSpeculativeDecodingMode (e.g. multi-token prediction). The modes match the --speculative_decoding flag of the LiteRT-LM CLI.
StatusCode
Mirror of LiteRtLmStatusCode (the canonical Google/absl status codes), minus Ok.
TokenUnionType
Wraps LiteRtLmTokenUnionType.

Functions§

api_version
Queries the C API version of the loaded library (litert_lm_get_api_version).
check_api_version
Verifies that the loaded library is compatible with the headers this crate was built against. The (cheap) library query is cached after the first success.
clear_last_error
Clears the calling thread’s last-error state (litert_lm_clear_last_error). Optional; see error_reporter.h.
extract_text
Extracts and concatenates just the plain text from a message JSON object of the shape {"role": "...", "content": [{"type": "text", "text": "..."}, ...]} – i.e. what Conversation::send_message or each stream chunk’s StreamChunk::text returns. Non-text content blocks are skipped. Returns an empty string if the JSON doesn’t match the expected shape, since callers usually just want “whatever text is in here, if any”.
last_error_message
Returns the last error message recorded by the C library on the calling thread, if any.
set_min_log_level
Sets the minimum log severity for the underlying C++ library. Call this once, before creating an Engine, to quiet (or increase) its stderr logging. It writes straight to stderr, independent of anything in Rust, so this is the only way to control it.

Type Aliases§

Result
Convenience alias for Result<T, litertlm_rs::Error>.