Skip to main content

TransformerConfig

spine_crypto

Type Alias TransformerConfig 

Source 
pub type TransformerConfig = TitansConfig;

Aliased Type§

pub struct TransformerConfig {
    pub embed_dim: usize,
    pub num_heads: usize,
    pub num_layers: usize,
    pub ff_dim: usize,
    pub max_seq_len: usize,
    pub memory_size: usize,
    pub seed: u64,
}

Fields§

§embed_dim: usize§num_heads: usize§num_layers: usize§ff_dim: usize§max_seq_len: usize§memory_size: usize

Size of persistent memory (number of memory tokens)

§seed: u64