forgellm-runtime 0.2.0

Minimal runtime for ForgeLLM (KV cache, sampling, tokenizer, API server)

Coverage
87.32%
62 out of 71 items documented0 out of 51 items with examples
Size
Source code size: 76.3 kB This is the summed size of all the files inside the crates.io package for this release.
Documentation size: 1.25 MB This is the summed size of all files generated by rustdoc for all configured targets
Ø build duration
this release: 1m 12s Average build duration of successful builds.
all releases: 1m 28s Average build duration of successful builds in releases after 2024-10-23.
Links
Homepage
sauravpanda/forge-llm
3 0 2
crates.io
Dependencies
Versions
Owners

ForgeLLM Runtime — minimal inference runtime.

Provides KV cache management, token sampling, and tokenizer integration for compiled models.