Module attention_context 

Attention-Weighted Context Assembly.

Scientific basis:

• Treisman (1980), “A feature-integration theory of attention” — cognitive resources are allocated proportionally to information density, not uniformly.
• Transformer Self-Attention (Vaswani et al., 2017) — weighted focus on informative regions.

Applied here: when assembling context from multiple search results, we:

1. Score each chunk’s information density (unique tokens / total tokens)
2. Detect redundancy between selected chunks (token-level overlap)
3. Allocate token budget proportionally to density, not uniformly
4. Aggressively truncate low-density chunks

Structs

ChunkDensity

Information density metrics for a code chunk.

Functions

attention_weighted_assembly

Assemble context with attention-weighted budget allocation.

compute_density

Compute information density for a chunk.

compute_redundancy

Compute pairwise redundancy between two chunks (0.0 = unique, 1.0 = identical).

truncate_to_budget

Truncate content to fit within a token budget (approximate by chars/4).