Crate kime_engine 

Session, scheduler, batching, the state memory cache, the answer cache and the tokenization cache. This is the in process API that the server, the CLI and the language bindings all sit on. See spec/07-engine.md and spec/11-serving.md.

What is here today is the compat path end to end: open a Laya checkpoint, lay each question out as Laya does, run the questions of one or many requests in shared batches on the CPU or a CUDA GPU, and build Laya’s answers from the logits. The scheduler that merges requests from many callers arrives with the server, so for now a call runs on the caller’s thread and callers share the device through a lock.

The kime crate re-exports all of it, and its docs hold a full example.

Modules

hub

Finding a model by name, in the Hugging Face cache layout so downloads made by Laya or huggingface_hub are reused and ours are reused by them.

Structs

Builder

Settings for Kime, from Kime::builder.

Decision

The future Kime::decide_async returns.

Kime

A loaded model on a device. Clones share it, and it can be used from any thread.

Memory

Bytes a model holds on its device.

Timing

Where the time of one Kime::decide_batch_timed call went.

Enums

Device

Where the model runs.

Error

What can go wrong.

Precision

The number format. The CPU computes in FP32 for both float settings.