Crate hermes_mal 

Model Architecture Language (MAL) for Hermes LLM

A composable DSL for defining LLM model architectures using pest parser.

Example MAL - Minimal inline style

model tiny {
    vocab_size: 32000
    max_seq_len: 2048
    hidden_size: 128
    num_layers: 4
    block: {
        attention: { num_heads: 4 }
        ffn: { hidden_dim: 512 }
    }
}

Example MAL - Composable style

# Define attention mechanism
attention gqa {
    num_heads: 32
    num_kv_heads: 8
    head_dim: 128
    position_encoding: rope { theta: 10000.0 }
}

# Define FFN
ffn swiglu_mlp {
    hidden_dim: 14336
    activation: swiglu
    bias: false
}

# Define transformer block
block llama_block {
    attention: gqa
    ffn: swiglu_mlp
    norm: rmsnorm { eps: 1e-5 }
    norm_position: pre
}

# Define model using the block
model llama_7b {
    vocab_size: 32000
    max_seq_len: 4096
    hidden_size: 4096
    block: llama_block
    num_layers: 32
}

Structs

AttentionDef

Attention mechanism definition

BlockDef

Transformer block definition

EmbeddingsConfig

Embeddings configuration

FfnDef

Feed-forward network definition

MalFile

Complete parsed MAL file with all definitions

MalParser

ModelDef

Parsed model definition from MAL

MoeDef

Configurable dropless token-choice mixture of experts.

NormConfig

Normalization configuration

OutputConfig

Output head configuration

SsmDef

Selective state-space (Mamba) mixer definition

Enums

Activation

Activation function

NormPosition

Normalization position in block

NormType

Normalization type

PositionEncoding

Position encoding type

Rule

Functions

get_builtin_model

Get a well-known model definition by name

get_wellknown_mal

Get the raw MAL content for a well-known model

list_wellknown_models

List all well-known model names (auto-discovered from embedded files)

parse_mal

Parse a MAL string containing exactly one model definition.

parse_mal_file

Parse MAL from a file

parse_mal_full

Parse complete MAL file with all definitions