Skip to main content

Module shape

Module shape 

Source

Structs§

KvTensorShape
Describes the tensor shape of a KV cache for a specific model family.

Enums§

AttentionType
The type of attention mechanism in the transformer model.