Module protocols 

Structs

ActiveBlockEvent

ActiveSequenceEvent

ExternalSequenceBlockHash

A sequence aware hash of a block where the hash is computed from the tokens_ids, extra_token_ids and the optional lora_id of a block, PLUS the hash of the parent block.

ForwardPassMetrics

KvCacheEvent

Represents a single cache event with an ID and associated data.

KvCacheEvents

Represents a collection of cache events and a shutdown flag.

KvCacheRemoveData

Represents the data associated with a removed cache event.

KvCacheStoreData

Represents the data associated with a stored cache event.

KvCacheStoredBlockData

Represents data for a stored block.

KvStats

LocalBlockHash

A LocalBlockHash is a hash computed from the tokens_ids, extra_token_ids and the optional lora_id of a block.

PredictiveLoadMetrics

PrefillEvent

SpecDecodeStats

WorkerSelectionResult

WorkerStats

Enums

ActiveBlockEventData

ActiveSequenceEventData

KvCacheEventData

Represents the data associated with a cache event.

LoadMetrics

PrefillEventData

Represents the different stages of prefilling tokens for a request.

RouterRequest

RouterResponse