litertlm 0.2.0

Safe, idiomatic Rust bindings to Google's LiteRT-LM on-device LLM engine.
docs.rs failed to build litertlm-0.2.0
Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.

litertlm

Safe Rust bindings to Google's LiteRT-LM — on-device LLM inference.

[dependencies]
litertlm = "0.2"

use litertlm::{Backend, Engine, EngineSettings, SamplerParams};

let engine = Engine::new(
    EngineSettings::new("model.litertlm")
        .backend(Backend::Gpu)
        .max_num_tokens(512),
)?;

let mut session = engine.create_session(
    SamplerParams::default().top_k(40).temperature(0.7),
)?;

let response = session.generate("Explain Rust lifetimes")?;
println!("{response}");
# Ok::<(), litert_lm::Error>(())

See the workspace README for the full story.

License

Apache-2.0; see NOTICE.