Struct GPTConfig 

pub struct GPTConfig {
    pub vocab_size: usize,
    pub context_length: usize,
    pub embed_dim: usize,
    pub num_layers: usize,
    pub num_heads: usize,
    pub ff_dim: usize,
    pub dropout: f32,
    pub bias: bool,
}

GPT configuration

Fields

vocab_size: usize

Vocabulary size

context_length: usize

Context length (maximum sequence length)

embed_dim: usize

Embedding dimension

num_layers: usize

Number of transformer layers

num_heads: usize

Number of attention heads

ff_dim: usize

Feed-forward hidden dimension

dropout: f32

Dropout probability

bias: bool

Use bias in linear layers

Implementations

impl GPTConfig

pub fn gpt2_small() -> Self

GPT-2 Small (117M parameters)

pub fn gpt2_medium() -> Self

GPT-2 Medium (345M parameters)

pub fn gpt2_large() -> Self

GPT-2 Large (774M parameters)

pub fn gpt2_xl() -> Self

GPT-2 XL (1.5B parameters)

pub fn gpt3_small() -> Self

GPT-3 Small (125M parameters)

pub fn gpt3_medium() -> Self

GPT-3 Medium (350M parameters)

pub fn gpt3_large() -> Self

GPT-3 Large (760M parameters)

pub fn gpt3_xl() -> Self

GPT-3 XL (1.3B parameters)

pub fn gpt_tiny() -> Self

GPT-Tiny (for testing)

Trait Implementations

impl Clone for GPTConfig

fn clone(&self) -> GPTConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GPTConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for GPTConfig

fn default() -> Self

Returns the “default value” for a type.