Expand description
This module contains protocol types and utilities used for communicating with llmvm core & backends.
Uses multilink to communicate with local & remote processes.
Modules§
- error
- Protocol error types.
- http
- jsonrpc
- JSON-RPC types and methods.
- service
- stdio
- tower
async fn(Request) -> Result<Response, Error>- util
- Miscellaneous utility functions.
Structs§
- Backend
Generation Request - Request for language model generation.
- Backend
Generation Response - Response for language model generation.
- Generation
Parameters - Parameters used for generation via core service.
Can be saved in a preset and/or directly provided within the
GenerationRequest. - Generation
Request - Request for text generation via core service.
- Generation
Response - Response for text generation via core service.
- Message
- A prompt or generated message from a thread.
- Model
Description - A parsed model id data structure.
- Protocol
Error - A “one size fits all” error type for the protocol. Contains a boxed error, and the error type.
- Thread
Info - Metadata for a thread.
Enums§
- Message
Role - The actor who presented the message.
- Service
Response - A response container returned by a multilink service.
Constants§
- CHAT_
MODEL_ PROVIDER_ SUFFIX - DEFAULT_
TIMEOUT_ SECS - Default request timeout.
Traits§
- Backend
- The backend service which the core uses to generate text. Implements a low-level interface for interacting with language models.
- Config
Example Snippet - A configuration data structure that provides an example for generating new TOML configuration files. The example should include customizable fields with comments explaining their purpose.
- Core
- The core service which frontends use to interact with language models. Manages & uses threads, presets, prompt templates and backend connections to create & send backend requests.
Type Aliases§
- Boxed
Service - A boxed dynamic type for multilink services. The service must return
a result with a
ServiceResponseorServiceError. - Notification
Stream - A stream of multiple response results returned by the service.
- Service
Error - A boxed error type that may be returned by service calls.
- Service
Future - A future that returns a result with a generic response and
ServiceError. This is returned by service calls.