Skip to main content

Module roster

Module roster 

Source
Expand description

Import of the model lab’s roster (~/llm/docs/models/queue.yaml, card mesh-placement-20; ADR-101 section 8 step 2) into InferenceSpecs.

The roster is read, never written, and ~/llm stays a separate repo: the caller names the file. What the roster holds maps directly (role id, backend, port, ram_gb); what it does not hold (co-residency excludes, latency class, stickiness) comes from an operator RosterOverlay, with the ADR’s defaults otherwise: inference is sticky (warm KV), roles are batch unless the overlay or the role’s text says voice. Entries the adapters cannot run (vision, embeddings, STT, TTS through mlx-audio) and aliases of an earlier entry are reported in ImportedRoster::skipped with the reason, not dropped silently.

Structs§

ImportedRoster
The result of an import.
ImportedSpec
One roster entry turned into a spec, with what was left behind.
RosterOverlay
Operator-supplied facts the roster does not carry.

Constants§

MAX_ROSTER_BYTES
Largest roster file read.

Functions§

flavor_for_backend
The adapter flavor the roster’s backend text names (the first listed when it offers alternatives).
import_roster
Parse roster YAML. overlay supplies excludes, latency and stickiness.