Expand description
Import of the model lab’s roster (~/llm/docs/models/queue.yaml, card
mesh-placement-20; ADR-101 section 8 step 2) into InferenceSpecs.
The roster is read, never written, and ~/llm stays a separate repo:
the caller names the file. What the roster holds maps directly (role id,
backend, port, ram_gb); what it does not hold (co-residency
excludes, latency class, stickiness) comes from an operator
RosterOverlay, with the ADR’s defaults otherwise: inference is sticky
(warm KV), roles are batch unless the overlay or the role’s text says
voice. Entries the adapters cannot run (vision, embeddings, STT, TTS
through mlx-audio) and aliases of an earlier entry are reported in
ImportedRoster::skipped with the reason, not dropped silently.
Structs§
- Imported
Roster - The result of an import.
- Imported
Spec - One roster entry turned into a spec, with what was left behind.
- Roster
Overlay - Operator-supplied facts the roster does not carry.
Constants§
- MAX_
ROSTER_ BYTES - Largest roster file read.
Functions§
- flavor_
for_ backend - The adapter flavor the roster’s
backendtext names (the first listed when it offers alternatives). - import_
roster - Parse roster YAML.
overlaysupplies excludes, latency and stickiness.