Skip to main content

Module pooling

Module pooling 

Source
Expand description

Pooling strategies for transformer hidden states.

Converts per-token hidden states [seq_len × dim] into a single fixed-size embedding vector [dim].

Functions§

l2_norm
Compute the L2 norm of a vector.
mean_pool
Mean pooling over token positions, weighted by attention mask.
normalize_l2
L2-normalize a vector in-place.