An egui app for prompting a local offline LLM.

Description

coze is a small egui application for prompting a local offline LLM using the Huggingface candle crate.

Currently it uses a quantized version of the StableLM 2 Zephyr 1.6B model that is a relatively small model that is fun to use.

The current version supports:

• Prompt history navigation with fuzzy matching.
• History persistence across runs.
• Token generation modes.
• Copy prompts and replies to clipboard.
• Light/Dark mode.

See the app Edit/Config menu for usage details.

Installation

The latest version of coze can be installed or updated with cargo install:

cargo install --locked coze

or by getting the binaries generated by the release Github action for Linux, macOS, and Windows in the releases page.

The first time it runs it will download the model weights from Huggingface to the ~/.cache/coze folder.

To build locally (debug build may be very slow):

git clone https://github.com/vincev/coze
cd coze
cargo r --release