Module opt 

OPT Model V2 - Clean implementation

OPT (Open Pre-trained Transformer) architecture features:

• Learned absolute position embeddings
• Standard multi-head attention
• Pre-norm layer normalization

Structs

OPTAttention

OPTConfig

OPTLayer

OPTModelV2