Expand description
§RPhi
RPhi is a Rust implementation of the quantized Phi 1.5 language model.
Phi-1.5 is a very small but performant language model that can be easily run on your local machine.
This library uses Quantized Mixformer from Candle to run Phi-1.5.
§Usage
use rphi::prelude::*;
#[tokio::main]
async fn main() {
let mut model = Phi::default();
let prompt = "The capital of France is ";
let mut result = model.stream_text(prompt).await.unwrap();
print!("{prompt}");
while let Some(token) = result.next().await {
print!("{token}");
}
}
Re-exports§
pub use kalosm_language_model;
Modules§
- A prelude of commonly used items in RPhi.
Structs§
- A quantized Phi-1.5 language model with support for streaming generation.
- A builder with configuration for a Phi model.
- A PhiSource is the source to fetch a Phi-1.5 model from. The model to use, check out available models: https://huggingface.co/models?other=mixformer-sequential&sort=trending&search=phi The model must have a quantized version available with a safetensors file. (for example lmz/candle-quantized-phi)