Module dbrx 

DBRX Model V2 - Clean implementation

DBRX (Databricks) architecture features:

• Fine-grained MoE with 16 experts, top-4 routing
• RoPE embeddings
• Grouped Query Attention

Structs

DbrxAttention

DbrxConfig

DbrxExpert

DbrxLayer

DbrxMoE

DbrxModelV2