pantry-rs 0.0.1

Client library for the Pantry LLM API.

It's strongly recommended that you use [PantryClient] and [LLMSession], which are a higher level wrapper around [PantryAPI].

let perms = UserPermissions {
perm_superuser: false,
perm_load_llm: false,
perm_unload_llm: false,
perm_download_llm: false,
perm_session: true, //this is for create_session AND prompt_session
perm_request_download: true,
perm_request_load: true,
perm_request_unload: true,
perm_view_llms: true,
};

let pantry = PantryClient::register("my project name".into(), perms).await.unwrap();

// Pause here and use the UI to accept the permission request.

// The empty hashmap means we just use default parameters.
// create_session just uses the best currently running LLM. use create_session_id or _flex for
// more finegrained control
let sess = pantry.create_session(HashMap::new()).await.unwrap();

let recv = ses.prompt_session("About me: ".into(), HashMap::new()).await.unwrap();

If you aren't already running an LLM from the ui, you can use

pantry.load_llm_flex(None, None).await.unwrap();

If you want to use your existing ggml infrastructure, you can get a bare model path

let (model, path) = pantry.bare_model_flex(None, None).await.unwrap();