rust_llama.cpp
LLama.cpp rust bindings.
The rust bindings are mostly based on https://github.com/go-skynet/go-llama.cpp/
Building Locally
Note: This repository uses git submodules to keep track of LLama.cpp.
Clone the repository locally:
Usage
[]
= "0.2.0"
use ;
Examples
The examples contain dockerfiles to run them
see examples
TODO
- Implement support for cublas,openBLAS & OpenCL #7
- Implement support for GPU (Metal)
- Add some test cases
- Support for fetching models through http & S3
- Sync with latest master & support GGUF
- Add some proper examples https://github.com/mdrokz/rust-llama.cpp/pull/7
LICENSE
MIT