Crate lex_analysis 

Document analysis and navigation for Lex

This crate provides semantic analysis capabilities for Lex documents, enabling features like reference resolution, symbol extraction, token classification, and document navigation.

Architecture

The crate is organized into several modules:

• utils: Core document traversal and lookup utilities
• references: Reference resolution and target conversion
• inline: Inline span detection (bold, italic, code, references)
• tokens: Semantic token extraction and classification
• symbols: Document structure and symbol hierarchy
• hover: Preview text extraction for hover tooltips
• folding: Foldable range detection
• navigation: Go-to-definition and find-references

Design Principles

• Stateless: All functions operate on immutable AST references
• Reusable: Not tied to LSP protocol - usable by CLI, editor plugins, etc.
• Well-tested: Comprehensive unit tests using official sample fixtures
• AST-focused: Works directly with lex-parser AST types

Usage

use lex_analysis::{tokens, symbols, navigation};
use lex_core::parse;

let document = parse("1. Introduction\n\nHello world")?;

// Extract tokens for syntax highlighting
let tokens = tokens::extract_semantic_tokens(&document);

// Build document outline
let symbols = symbols::extract_document_symbols(&document);

// Resolve references
let defs = navigation::find_definition(&document, position);

Modules

annotations

Annotation navigation and resolution editing.

completion

Context-aware completion for Lex documents.

diagnostics

document_symbols

folding_ranges

go_to_definition

hover

inline

reference_targets

references

semantic_tokens

This is the semantic token collector, which editors use for syntax highlighting. It’s worth going over the general approach.

spellcheck

Spellcheck analysis for Lex documents.

utils