Crate pantry_rs

Client library for the Pantry LLM API.

It’s strongly recommended that you use PantryClient and LLMSession, which are a higher level wrapper around PantryAPI.

let perms = UserPermissions {
    perm_superuser: false,
    perm_load_llm: false,
    perm_unload_llm: false,
    perm_download_llm: true,
    perm_session: true, //this is for create_session AND prompt_session
    perm_request_download: true,
    perm_request_load: true,
    perm_request_unload: true,
    perm_view_llms: true,
};

let pantry = PantryClient::register("my project name".into(), perms).await.unwrap();

// Pause here and use the UI to accept the permission request.

// The empty hashmap means we just use default parameters.
// create_session just uses the best currently running LLM. use create_session_id or _flex for
// more finegrained control
let sess = pantry.create_session(HashMap::new()).await.unwrap();

let recv = ses.prompt_session("About me: ".into(), HashMap::new()).await.unwrap();

If you aren’t already running an LLM from the ui, you can use

pantry.load_llm_flex(None, None).await.unwrap();

If you want to use your existing ggml infrastructure, you can get a bare model path

let (model, path) = pantry.bare_model_flex(None, None).await.unwrap();

Re-exports

pub use self::error::PantryError;

pub use api::PantryAPI;

pub use api::LLMFilter;

pub use api::LLMPreference;

Modules

api

Low Level API Wrapper

error

interface

Structs

LLMSession

PantryClient

Wrapper around the Pantry LLM API.