RPhi

RPhi is a Rust implementation of the quantized Phi 1.5 language model.

Phi-1.5 is a very small but performant language model that can be easily run on your local machine.

This library uses Quantized Mixformer from Candle to run Phi-1.5.

Usage

use rphi::prelude::*;
#[tokio::main]
async fn main() {
let mut model = Phi::default();
let prompt = "The capital of France is ";
let mut result = model.stream_text(prompt).await.unwrap();

print!("{prompt}");
while let Some(token) = result.next().await {
print!("{token}");
}
}