Expand description
Finding models already on disk. This module holds the format-agnostic pieces — multi-part GGUF shard grouping and duplicate-weight detection. The directory scanners and reconciliation into the registry build on these.
Re-exports§
pub use duplicates::DEFAULT_THRESHOLD;pub use duplicates::DuplicateGroup;pub use duplicates::content_fingerprint;pub use duplicates::detect;pub use gguf_models::discovered_models;pub use gguf_models::is_mmproj_name;pub use gguf_shards::Member;pub use gguf_shards::ShardGroup;pub use gguf_shards::ShardName;pub use gguf_shards::group;pub use gguf_shards::parse;pub use gguf_shards::shard_filename;pub use habitat::ModelHabitat;pub use habitat::ModelsSettings;pub use habitat::hf_cache_root;pub use habitat::hf_home;pub use hf_scanner::HFCacheScanner;pub use lm_studio_scanner::LMStudioScanner;pub use loose_file_scanner::LooseFileScanner;pub use modality_hints::Hint;pub use ollama_scanner::OllamaStoreScanner;pub use scanner::DiscoveredModel;pub use scanner::ScanResult;pub use scanner::StoreScanner;pub use service::DiscoveryService;pub use service::DiscoverySummary;pub use service::KindStat;
Modules§
- duplicates
- Detecting duplicate model weights by size then a cheap content fingerprint.
- gguf_
models - Turning a flat list of GGUF files into
DiscoveredModels: loose files become one model each, shard sets become a single model keyed by their shared base (flagged as still-downloading when incomplete). Shared by the file-tree scanners (LM Studio, loose files). - gguf_
shards - Recognizing and grouping multi-part GGUF weight files named like
model-00001-of-00005.gguf. - habitat
ModelHabitat: where models live on this machine. It computes the store roots (from the environment, the home directory, and user settings) and assembles theStoreScanners that sweep them. This is the single place the four scanners are wired together and pointed at their default locations.- hf_
scanner - Scans a Hugging Face hub cache: each
models--<org>--<repo>directory’s current snapshot is inspected (config.json/model_index.json/ a bare GGUF) for a modality hint, its blobs summed for the footprint, and its shard/blob completeness checked to flag a still-downloading model. - lm_
studio_ scanner - Scans LM Studio’s model tree: each root is walked for
.ggufweight files (skipping multimodal projectors), which are grouped into models bydiscovered_models. The repo label is the<publisher>/<model>prefix of a file’s path relative to the root. - loose_
file_ scanner - Scans loose directories (Downloads, a Models folder) up to a shallow depth
for models a user dropped in by hand: a
config.json+safetensorsfolder becomes one bundle model, loose.gguffiles group into models, and a GGML-magic.binis a whisper transcription model. - modality_
hints - Modality/capability hints derived from a Hugging Face model’s
config.json: a first guess (architecture list, a vision block, or a few speech config keys) at what a downloaded model can do, before full identification. A diffusersmodel_index.jsonis hinted through the pipeline-family registry (from_model_index). - ollama_
scanner - Scans a local Ollama store (
~/.ollama/models): walksmanifests/<registry>/ <namespace>/<model>/<tag>, reads each manifest’s layer list, and resolves the weight/template/projector/params blobs into aDiscoveredModel. - scanner
- The shared discovery surface: a
StoreScannerinspects one on-disk model store (an Ollama blob store, a Hugging Face cache, …) and returns the models it found asDiscoveredModelhints, plus any per-store issues, in aScanResult. Identification later turns these hints into full records. - service
DiscoveryService: the discovery capstone. It runs the store scanners and reconciles what they found against the registry — updating known records, registering new ones, marking on-disk-gone models missing (guarded by a weights-present check), migrating a moved model’s saved config onto its new record, and summarizing the result.