Skip to main content

Module attention

Module attention 

Source
Expand description

NAPI-RS bindings for attention mechanisms

Provides Node.js bindings for all attention variants:

  • Scaled dot-product attention
  • Multi-head attention
  • Hyperbolic attention
  • Flash attention
  • Linear attention
  • Local-global attention
  • Mixture of Experts attention

Structs§

AttentionConfig
Attention configuration object
DotProductAttention
Scaled dot-product attention
FlashAttention
Flash attention with tiled computation
HyperbolicAttention
Hyperbolic attention in Poincaré ball model
LinearAttention
Linear attention (Performer-style) with O(n) complexity
LocalGlobalAttention
Local-global attention (Longformer-style)
MoEAttention
Mixture of Experts attention
MoEConfig
MoE attention configuration
MultiHeadAttention
Multi-head attention mechanism

Functions§

exp_map
Exponential map from tangent space to hyperbolic space
log_map
Logarithmic map from hyperbolic space to tangent space
mobius_addition
Möbius addition in hyperbolic space
poincare_distance
Compute hyperbolic (Poincaré) distance between two points
project_to_poincare_ball
Project a vector into the Poincaré ball