tensorlogic-trustformers 0.1.0

Transformer-as-rules: Self-attention and FFN layers as einsum expressions
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

//! Sparse attention patterns and Longformer-style numerical attention.
//!
//! This module provides two layers:
//!
//! 1. **Graph-building** (`graph`) — the original einsum-graph sparse-attention
//!    types (`SparseAttentionGraph`, `SparseAttentionGraphConfig`, `LocalAttention`,
//!    `SparsePatternType`).
//!
//! 2. **Numerical Longformer attention** (`attention`, `config`, `mask`, `error`)
//!    — a research-preview implementation of sliding-window + global-token
//!    attention (Beltagy et al., 2020) that performs actual tensor computation
//!    rather than building an einsum graph.

pub mod attention;
pub mod config;
pub mod error;
pub mod graph;
pub mod mask;

#[cfg(test)]
mod tests;

// ---- Numerical sparse attention types (Longformer-style) ----
pub use attention::SparseAttention;
pub use config::SparseAttentionConfig;
pub use error::{SparseAttentionError, SparseAttentionResult};
pub use mask::{build_mask, AttentionMask};

// ---- Graph-building types (original) ----
pub use graph::{
    LocalAttention, SparseAttentionGraph, SparseAttentionGraphConfig, SparsePatternType,
};