Skip to main content

Module local_llm

Module local_llm 

Source
Expand description

Location and accessors for the bundled local-model base URL.

Hub starts an MLX inference server on an ephemeral port and writes its OpenAI-compatible base URL here. Agents — started by launchd and therefore NOT inheriting Hub’s environment — read it back from this file.

Constants§

DEFAULT_LOCAL_MODEL_DIR
Directory name (under <mur_home>/models/).
DEFAULT_LOCAL_MODEL_REPO
HuggingFace repo holding bundled-free local model.

Functions§

base_url_path
Path to the file holding the local model base URL, under <mur_home>.
local_model_dir
Path to the local model directory under <mur_home>/models/<model_dir>.
model_complete_marker
Path to the model-complete marker file (.complete) within the model directory.
read_base_url
Read the base URL, trimming whitespace. None if absent/empty.
write_base_url
Atomically write the base URL (temp file + rename).