Module protocols 

Dynamo LLM Protocols

This module contains the protocols, i.e. messages formats, used to exchange requests and responses both publicly via the HTTP API and internally between Dynamo components.

Modules

codec

A module for parsing Server-Sent Events (SSE) streams according to the SSE specification.

common

Engine Protocols

openai

tensor

Structs

Annotated

Our services have the option of returning an “annotated” stream, which allows use to include additional information with each delta. This is useful for debugging, performance benchmarking, and improved observability.

Traits

ContentProvider

The LLM responses have multiple different fields and nests of objects to get to the actual text completion returned. This trait can be applied to the choice level objects to extract the completion text.

Functions

convert_sse_stream

Converts of a stream of codec::Messages into a stream of Annotateds.

Type Aliases

DataStream

TokenIdType

The token ID type