Skip to main content

Module discovery

Module discovery 

Source
Expand description

Finding models already on disk. This module holds the format-agnostic pieces — multi-part GGUF shard grouping and duplicate-weight detection. The directory scanners and reconciliation into the registry build on these.

Re-exports§

pub use duplicates::DEFAULT_THRESHOLD;
pub use duplicates::DuplicateGroup;
pub use duplicates::content_fingerprint;
pub use duplicates::detect;
pub use gguf_models::discovered_models;
pub use gguf_models::is_mmproj_name;
pub use gguf_shards::Member;
pub use gguf_shards::ShardGroup;
pub use gguf_shards::ShardName;
pub use gguf_shards::group;
pub use gguf_shards::parse;
pub use gguf_shards::shard_filename;
pub use habitat::ModelHabitat;
pub use habitat::ModelsSettings;
pub use habitat::hf_cache_root;
pub use habitat::hf_home;
pub use hf_scanner::HFCacheScanner;
pub use lm_studio_scanner::LMStudioScanner;
pub use loose_file_scanner::LooseFileScanner;
pub use modality_hints::Hint;
pub use ollama_scanner::OllamaStoreScanner;
pub use scanner::DiscoveredModel;
pub use scanner::ScanResult;
pub use scanner::StoreScanner;
pub use service::DiscoveryService;
pub use service::DiscoverySummary;
pub use service::KindStat;

Modules§

duplicates
Detecting duplicate model weights by size then a cheap content fingerprint.
gguf_models
Turning a flat list of GGUF files into DiscoveredModels: loose files become one model each, shard sets become a single model keyed by their shared base (flagged as still-downloading when incomplete). Shared by the file-tree scanners (LM Studio, loose files).
gguf_shards
Recognizing and grouping multi-part GGUF weight files named like model-00001-of-00005.gguf.
habitat
ModelHabitat: where models live on this machine. It computes the store roots (from the environment, the home directory, and user settings) and assembles the StoreScanners that sweep them. This is the single place the four scanners are wired together and pointed at their default locations.
hf_scanner
Scans a Hugging Face hub cache: each models--<org>--<repo> directory’s current snapshot is inspected (config.json / model_index.json / a bare GGUF) for a modality hint, its blobs summed for the footprint, and its shard/blob completeness checked to flag a still-downloading model.
lm_studio_scanner
Scans LM Studio’s model tree: each root is walked for .gguf weight files (skipping multimodal projectors), which are grouped into models by discovered_models. The repo label is the <publisher>/<model> prefix of a file’s path relative to the root.
loose_file_scanner
Scans loose directories (Downloads, a Models folder) up to a shallow depth for models a user dropped in by hand: a config.json+safetensors folder becomes one bundle model, loose .gguf files group into models, and a GGML-magic .bin is a whisper transcription model.
modality_hints
Modality/capability hints derived from a Hugging Face model’s config.json: a first guess (architecture list, a vision block, or a few speech config keys) at what a downloaded model can do, before full identification. A diffusers model_index.json is hinted through the pipeline-family registry (from_model_index).
ollama_scanner
Scans a local Ollama store (~/.ollama/models): walks manifests/<registry>/ <namespace>/<model>/<tag>, reads each manifest’s layer list, and resolves the weight/template/projector/params blobs into a DiscoveredModel.
scanner
The shared discovery surface: a StoreScanner inspects one on-disk model store (an Ollama blob store, a Hugging Face cache, …) and returns the models it found as DiscoveredModel hints, plus any per-store issues, in a ScanResult. Identification later turns these hints into full records.
service
DiscoveryService: the discovery capstone. It runs the store scanners and reconciles what they found against the registry — updating known records, registering new ones, marking on-disk-gone models missing (guarded by a weights-present check), migrating a moved model’s saved config onto its new record, and summarizing the result.