Skip to main content

Module models

Module models 

Source
Expand description

Registry of model providers and the models they offer.

Providers are stored in a map keyed by their stable unique id (e.g. "bedrock"). Models are identified globally by a ModelRef (provider id + provider-scoped model id).

Providers run as separate processes configured in ~/.infinity/providers.json (see ProvidersConfig): each entry maps a provider id to the command that serves that provider over a Unix socket. The daemon spawns each command, reads the socket path the process prints on stdout, and talks to it through a RemoteModelProvider.

Structs§

CatalogModel
A model in the catalog, tagged with the provider that offers it.
CatalogModelSource
[ModelSource] over the daemon’s catalog: each completion round resolves the model persisted for its thread in the conversation store, falling back to the global default when the selection is no longer available. Model switching is just persisting a new selection — the next round picks it up, and an in-flight completion finishes on the model it started with.
ModelCatalog
All registered providers and their available models.
ProviderConfig
Configuration for a single model provider process in providers.json.
ProvidersConfig
The providers configured in ~/.infinity/providers.json: a JSON object mapping provider id to a ProviderConfig, e.g.:

Functions§

load_providers_config
Load the providers config from path. There is no implicit default: at least one provider must be configured (e.g. via infinity provider install) for the daemon to start.
spawn_configured_providers
Spawn every provider in the config. Returns the (provider_id, provider) pairs for ModelCatalog::new plus the child process handles, which must be kept alive for as long as the providers are in use.
spawn_provider
Spawn a provider process and wait for it to print its socket path on stdout. The returned Child kills the process when dropped; callers must keep it alive for as long as the provider is in use.