Module chatglm 

ChatGLM Model V2 - Clean implementation using solid abstractions

This implements the ChatGLM architecture including:

• ChatGLM-6B, ChatGLM2-6B, ChatGLM3-6B, GLM-4

ChatGLM unique characteristics:

• Structure: embedding -> encoder -> output_layer
• Uses transformer.embedding.word_embeddings.weight
• Uses transformer.encoder.layers.{i}
• Uses transformer.output_layer.weight
• Packed QKV attention (query_key_value combined)
• SwiGLU activation in MLP

Structs

ChatGLMAttention

ChatGLM attention mechanism with packed QKV

ChatGLMConfig

ChatGLMLayer

ChatGLM transformer layer

ChatGLMMLP

ChatGLM MLP with SwiGLU activation

ChatGLMModelV2

Main ChatGLM model implementation