1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
//! Language Server Protocol (LSP) implementation for Lex
//!
//! This crate provides language server capabilities for the Lex format, enabling rich editor
//! support in any LSP-compatible editor (VSCode, Neovim, Emacs, Sublime, etc.).
//!
//! Design Decision: tower-lsp
//!
//! After evaluating the Rust LSP ecosystem, we chose tower-lsp as our framework:
//!
//! Considered Options:
//! 1. tower-lsp: High-level async framework built on Tower (~204K monthly downloads)
//! 2. lsp-server: Low-level sync library from rust-analyzer (~309K monthly downloads)
//! 3. async-lsp: Low-level async with full Tower integration (~135 GitHub stars)
//!
//! Why tower-lsp:
//! - Best balance of ease-of-use and functionality for a new LSP project
//! - Strong ecosystem support with extensive documentation and examples
//! - Modern async/await patterns ideal for Lex's structured parsing needs
//! - Built-in LSP 3.18 support with proposed features
//! - Active community with production usage in many language servers
//! - Good integration with Rust async ecosystem (tokio, futures)
//!
//! Trade-offs:
//! - Less flexible than async-lsp for custom Tower layers (acceptable for our needs)
//! - Requires &self for trait methods, forcing Arc<Mutex<>> for mutable state (standard pattern)
//! - Notification ordering is async (not an issue for initial feature set)
//!
//! Future Migration Path:
//! If we later need precise notification ordering or custom middleware, we can migrate
//! to async-lsp with minimal disruption as both use similar async patterns.
//!
//! Feature Set
//!
//! Lex is a structured document format, not a programming language. LSP features are selected
//! to optimize document authoring and navigation workflows:
//!
//! Core Features:
//!
//! a. Syntax Highlighting
//! 1. Semantic Tokens (textDocument/semanticTokens/*):
//! - Syntax highlighting for sessions, lists, definitions, annotations
//! - Inline formatting: bold, italic, code, math
//! - References, footnotes, citations
//! - Verbatim blocks with language-specific highlighting
//! 2. Document Symbols (textDocument/documentSymbol):
//! - Hierarchical outline view of document structure
//! - Sessions with nesting (1., 1.1., 1.1.1., etc.)
//! - Definitions, annotations, lists as navigable symbols
//! 3. Hover Information (textDocument/hover):
//! - Preview footnote/citation content on hover
//! - Show annotation metadata
//! - Preview definition content when hovering over reference
//! 4. Folding Ranges (textDocument/foldingRange):
//! - Fold sessions and nested content
//! - Fold list items with children
//! - Fold annotations, definitions, verbatim blocks
//!
//! b. Navigation
//!
//! 5. Go to Definition / Find References (textDocument/definition, textDocument/references):
//! - Find all references to footnotes/citations
//! - Jump from footnote reference [42] to annotation
//! - Jump from citation [@spec2025] to bibliography entry
//! - Jump from internal reference [TK-rootlist] to target
//! 6. Document Links (textDocument/documentLink):
//! - Clickable links in text
//! - Verbatim block src parameters (images, includes)
//! - External references
//!
//! c. Editing
//!
//! 7. Document Formatting (textDocument/formatting, textDocument/rangeFormatting):
//! - Fix indentation issues
//! - Normalize blank lines
//! - Align list markers //!
//!
//!
//!
//! Architecture
//!
//! The server follows a layered architecture:
//!
//! LSP Layer (tower-lsp):
//! - Handles JSON-RPC communication
//! - Protocol handshaking and capability negotiation
//! - Request/response routing
//!
//! Server Layer (this crate):
//! - Implements LanguageServer trait
//! - Manages document state and parsing
//! - Coordinates feature implementations
//! - Very thing, mostly calls the the feature layers over lex-parser
//! - Thin tests just asserting the right things are being called and returned
//!
//! Feature Layer:
//! - Each feature operates on Lex AST
//! - Stateless transformations where possible
//! - All logic and dense unit tests
//!
//!
//! Testing Strategy
//!
//! Following Lex project conventions:
//! - Use official sample files from specs/ for all tests
//! - Use lexplore loader for consistent test data
//! - Use ast_assertions library for AST validation
//! - Test each feature in isolation and integration
//! - Test against kitchensink and trifecta fixtures
//!
//! Non-Features
//!
//! The following LSP features are intentionally excluded as they don't apply to document formats:
//! - Code Lens: Not applicable to documents
//! - Type Hierarchy: No type system
//! - Implementation: No interfaces/implementations
//! - Moniker: For cross-repo linking, not needed
//! - Linked Editing Range: For paired tags (HTML/XML)
//! - Diagnostics: we don't have a clear vision for how that would work.
//!
//! Error Handling and Robustness
//!
//! The server is designed to be highly robust and crash-resistant, following these principles:
//!
//! 1. No Panics:
//! - We strictly avoid `unwrap()` and `expect()` in production code paths.
//! - All potential failure points (parsing, serialization, IO) return `Result`.
//! - Errors are propagated up the stack and handled gracefully.
//!
//! 2. Graceful Degradation:
//! - If a feature fails (e.g., semantic tokens calculation), we log the error and return
//! an empty result or `None` rather than crashing the server.
//! - This ensures that a bug in one feature doesn't bring down the entire editor experience.
//!
//! 3. Error Propagation:
//! - The `lex-parser` crate returns `Result` types for all parsing operations.
//! - The `lex-lsp` server maps these internal errors to appropriate LSP error codes
//! (e.g., `InternalError`, `InvalidRequest`) when communicating with the client.
//!
//! 4. Property-Based Testing:
//! - We use `proptest` to fuzz the server with random inputs (commands, document text)
//! to uncover edge cases and ensure stability under unexpected conditions.
//!
//! Usage
//!
//! This crate provides both a library and binary:
//!
//! Library:
//! ```rust
//! use lex_lsp::LexLanguageServer;
//! use tower_lsp::Server;
//!
//! #[tokio::main]
//! async fn main() {
//! let stdin = tokio::io::stdin();
//! let stdout = tokio::io::stdout();
//!
//! let (service, socket) = LspService::new(|client| LexLanguageServer::new(client));
//! Server::new(stdin, stdout, socket).serve(service).await;
//! }
//! ```
//!
//! Binary:
//! $ lex-lsp
//! Starts the language server on stdin/stdout for editor integration.
//!
pub use LexLanguageServer;