lexd_lsp/

lib.rs

//! Language Server Protocol (LSP) implementation for Lex
//!
//!     This crate provides language server capabilities for the Lex format, enabling rich editor
//!     support in any LSP-compatible editor (VSCode, Neovim, Emacs, Sublime, etc.).
//!
//! Position Encoding — read before touching any `Position`
//!
//!     In this server, `Position.character` is a **UTF-8 byte offset** into the line —
//!     NOT a UTF-16 code unit (the LSP default) nor a `char` count. This is the
//!     load-bearing convention every position↔text conversion here must honor.
//!
//!     It originates upstream: lex-core's `SourceLocation::byte_to_position` computes
//!     `column = byte_offset - line_start`, and `to_lsp_position` forwards that value to
//!     LSP unchanged. Every consumer of a `Position` in this crate must therefore read
//!     `.character` back as a byte offset to stay consistent.
//!
//!     Practical consequence: to take the text up to a caret, slice on the byte offset
//!     (`line.get(..pos.character as usize)`, which returns `None` rather than panicking
//!     on an out-of-bounds or mid-`char` index) — never
//!     `line.chars().take(pos.character as usize)`.
//!     A `char`-based count silently over-reads past the caret on any line containing
//!     multi-byte characters (this caused a real bug in #740). See `slice_text_by_range`
//!     in `server.rs` for the canonical byte-offset slicing routine, including the
//!     multi-byte-boundary guards.
//!
//!     Caveat — not yet uniform: `indent_level_from_position` in `server.rs` still uses
//!     the `chars().take()` form. It is currently harmless there (it only consumes leading
//!     ASCII indentation, where byte and `char` counts coincide), but it does not yet
//!     follow this convention and is a separate cleanup, not a reason to copy the pattern.
//!
//! Design Decision: tower-lsp
//!
//!     After evaluating the Rust LSP ecosystem, we chose tower-lsp as our framework:
//!
//!     Considered Options:
//!         1. tower-lsp: High-level async framework built on Tower (~204K monthly downloads)
//!         2. lsp-server: Low-level sync library from rust-analyzer (~309K monthly downloads)
//!         3. async-lsp: Low-level async with full Tower integration (~135 GitHub stars)
//!
//!     Why tower-lsp:
//!         - Best balance of ease-of-use and functionality for a new LSP project
//!         - Strong ecosystem support with extensive documentation and examples
//!         - Modern async/await patterns ideal for Lex's structured parsing needs
//!         - Built-in LSP 3.18 support with proposed features
//!         - Active community with production usage in many language servers
//!         - Good integration with Rust async ecosystem (tokio, futures)
//!
//!     Trade-offs:
//!         - Less flexible than async-lsp for custom Tower layers (acceptable for our needs)
//!         - Requires &self for trait methods, forcing Arc<Mutex<>> for mutable state (standard pattern)
//!         - Notification ordering is async (not an issue for initial feature set)
//!
//!     Future Migration Path:
//!         If we later need precise notification ordering or custom middleware, we can migrate
//!         to async-lsp with minimal disruption as both use similar async patterns.
//!
//! Feature Set
//!
//!     Lex is a structured document format, not a programming language. LSP features are selected
//!     to optimize document authoring and navigation workflows:
//!
//!     Core Features:
//!
//!         a. Syntax Highlighting

//!             1. Semantic Tokens (textDocument/semanticTokens/*):
//!                 - Syntax highlighting for sessions, lists, definitions, annotations
//!                 - Inline formatting: bold, italic, code, math
//!                 - References, footnotes, citations
//!                 - Verbatim blocks with language-specific highlighting
//!             2. Document Symbols (textDocument/documentSymbol):
//!                 - Hierarchical outline view of document structure
//!                 - Sessions with nesting (1., 1.1., 1.1.1., etc.)
//!                 - Definitions, annotations, lists as navigable symbols
//!            3. Hover Information (textDocument/hover):
//!                 - Preview footnote/citation content on hover
//!                 - Show annotation metadata
//!                 - Preview definition content when hovering over reference
//!             4. Folding Ranges (textDocument/foldingRange):
//!                 - Fold sessions and nested content
//!                 - Fold list items with children
//!                 - Fold annotations, definitions, verbatim blocks
//!
//!         b. Navigation
//!
//!             5. Go to Definition / Find References (textDocument/definition, textDocument/references):
//!                 - Find all references to footnotes/citations
//!                 - Jump from footnote reference [42] to annotation
//!                 - Jump from citation [@spec2025] to bibliography entry
//!                 - Jump from internal reference [TK-rootlist] to target
//!             6. Document Links (textDocument/documentLink):
//!                 - Clickable links in text
//!                 - Verbatim block src parameters (images, includes)
//!                 - External references
//!
//!         c. Editing
//!
//!             7. Document Formatting (textDocument/formatting, textDocument/rangeFormatting):
//!                 - Fix indentation issues
//!                 - Normalize blank lines
//!                 - Align list markers //!         
//!
//!
//!
//! Architecture
//!
//!     The server follows a layered architecture:
//!
//!     LSP Layer (tower-lsp):
//!         - Handles JSON-RPC communication
//!         - Protocol handshaking and capability negotiation
//!         - Request/response routing
//!
//!     Server Layer (this crate):
//!         - Implements LanguageServer trait
//!         - Manages document state and parsing
//!         - Coordinates feature implementations
//!         - Very thing, mostly calls the the feature layers over lex-parser
//!         - Thin tests just asserting the right things are being called and returned
//!
//!     Feature Layer:
//!         - Each feature operates on Lex AST
//!         - Stateless transformations where possible
//!         - All logic and dense unit tests
//!
//!
//! Testing Strategy
//!
//!     Following Lex project conventions:
//!         - Use official sample files from specs/ for all tests
//!         - Use lexplore loader for consistent test data
//!         - Use ast_assertions library for AST validation
//!         - Test each feature in isolation and integration
//!         - Test against kitchensink and trifecta fixtures
//!
//! Non-Features
//!
//!     The following LSP features are intentionally excluded as they don't apply to document formats:
//!         - Code Lens: Not applicable to documents
//!         - Type Hierarchy: No type system
//!         - Implementation: No interfaces/implementations
//!         - Moniker: For cross-repo linking, not needed
//!         - Linked Editing Range: For paired tags (HTML/XML)
//!         - Diagnostics: we don't have a clear vision for how that would work.
//!
//! Error Handling and Robustness
//!
//!     The server is designed to be highly robust and crash-resistant, following these principles:
//!
//!     1. No Panics:
//!         - We strictly avoid `unwrap()` and `expect()` in production code paths.
//!         - All potential failure points (parsing, serialization, IO) return `Result`.
//!         - Errors are propagated up the stack and handled gracefully.
//!
//!     2. Graceful Degradation:
//!         - If a feature fails (e.g., semantic tokens calculation), we log the error and return
//!           an empty result or `None` rather than crashing the server.
//!         - This ensures that a bug in one feature doesn't bring down the entire editor experience.
//!
//!     3. Error Propagation:
//!         - The `lex-parser` crate returns `Result` types for all parsing operations.
//!         - The `lexd-lsp` server maps these internal errors to appropriate LSP error codes
//!           (e.g., `InternalError`, `InvalidRequest`) when communicating with the client.
//!
//!     4. Property-Based Testing:
//!         - We use `proptest` to fuzz the server with random inputs (commands, document text)
//!           to uncover edge cases and ensure stability under unexpected conditions.
//!
//! Usage
//!
//!     This crate provides both a library and binary:
//!
//!     Library:
//!         ```rust
//!         use lexd_lsp::LexLanguageServer;
//!         use tower_lsp::Server;
//!
//!         #[tokio::main]
//!         async fn main() {
//!             let stdin = tokio::io::stdin();
//!             let stdout = tokio::io::stdout();
//!
//!             let (service, socket) = LspService::new(|client| LexLanguageServer::new(client));
//!             Server::new(stdin, stdout, socket).serve(service).await;
//!         }
//!         ```
//!
//!     Binary:
//!         $ lexd-lsp
//!         Starts the language server on stdin/stdout for editor integration.
//!

pub mod extension_dispatch;
pub mod features;
pub mod server;
pub mod trust_prompt;

pub use server::LexLanguageServer;