The model
A language plugs in twice; the server does the rest.
- A
Grammar(fromincremental-lang) parses each open document, and declares which nodes may be reparsed on their own. - A
Handleris the language server's behaviour. Every method has a default; a server implements only what its language supports, and lists what it offers inFeatures, from which the server's capabilities are advertised. - The
Serverruns the protocol over anyReadandWrite— stdin and stdout for an editor (serve_stdio), a buffer in a test — and calls the handler. - A
Contextis what the handler reaches while it runs: the open **Document**s, and a channel back to the client for notifications, requests, log messages, and diagnostics. - Parameters are **
Value**s fromjsonx-lib; results are typed (Hover,Location,CompletionItem,DocumentSymbol,SemanticTokens) or aValue, and a failure is aResponseError.
What the server owns, and what it hands to the handler:
| Message | Server | Handler |
|---|---|---|
initialize |
negotiates the position encoding; advertises document sync and the listed features | initialize reads the client's options; features says what is offered |
textDocument/didOpen, didChange |
applies the text, reparses incrementally (falling back to a full parse), patches the line index; never drops a document on a failed change | changed analyses and publishes diagnostics |
textDocument/didClose |
drops the document | closed (default: clears its diagnostics) |
hover, definition, completion, documentSymbol, semanticTokens/full and /delta |
reads the request, finds the document, writes the answer; the delta is computed from the previous answer | hover, definition, completion, document_symbols, semantic_tokens return typed results |
$/cancelRequest |
answers a request that has not started with RequestCancelled; the handler never sees it |
— |
| a registered custom request | answers protocol errors (before initialize, after shutdown) |
request returns the result |
| any other notification | drops it outside the session window | notification |
| an answer from the client | delivers it if it answers an outstanding request, once; logs anything else; cancels the oldest beyond 1024 outstanding | response |
shutdown, exit |
ends the session | — |
Installation
[]
= "2"
Or from the terminal:
incremental_lang, diag_lang, jsonx_lib, and rpc_io are re-exported, so a server needs no other dependency to write its grammar, build diagnostics, read parameters, or name a transport type.
Quick start
A complete server that warns about every TODO in a document. The grammar treats a whole document as one token; a real language supplies its own.
use ;
use ;
use ;
;
;
Offering features
List what the handler offers in features, and override the matching methods. The server advertises exactly the listed capabilities, reads each request, finds the document, calls the method, and writes the answer; positions arrive in the session encoding, and Document::offset turns one into a byte offset.
use Grammar;
use ;
/// Hover shows the character under the cursor and its byte offset.
;
Semantic tokens
A SemanticTokensBuilder turns byte spans from the syntax tree into the protocol's delta-encoded integers: it sorts, drops overlaps, converts to lines and columns in the session encoding (a surrogate pair counts as two in UTF-16), and splits a token that crosses a line break. The server keeps the last answer per document and answers a full/delta request with a single token-aligned edit.
use ;
use ;
use ;
;
Positions
The protocol counts columns in UTF-16 code units unless the client offers something else; the family counts byte offsets. A Document converts between them in the encoding negotiated at initialize — UTF-8 whenever the client offers it, because byte columns convert without scanning, and UTF-16 otherwise.
| Method | Converts | Use it for |
|---|---|---|
Document::offset |
Position → byte offset |
a request's cursor, to look up in the tree |
Document::span |
Range → Span |
a request's selection |
Document::position |
byte offset → Position |
a result's location |
Document::range |
Span → Range |
a tree node or a diagnostic |
Line breaks are \n, \r\n, and a lone \r, as the protocol specifies. Out-of-range input is clamped the way the protocol prescribes — a column past the end of its line means the end of the line — and a column inside a character (between the halves of a surrogate pair) snaps to that character's start. Conversions never fail and never panic.
Diagnostics
Context::publish_diagnostics takes diag_lang::Diagnostics whose spans are byte offsets into the document — what the syntax tree's spans already are — and writes textDocument/publishDiagnostics directly, with the document's version:
| diag-lang | Protocol |
|---|---|
| primary label's span | range |
Error, Warning, Note, Help |
severity 1 (Error), 2 (Warning), 3 (Information), 4 (Hint) |
| message, primary label, notes, help | message: the headline first, then one line each (note: …, help: …) |
| secondary labels | relatedInformation, located in the same document |
Diagnostics from symbol-lang's unresolved_name and duplicate_definition, or from any other family crate that reports through diag-lang, publish with no adapter; the server example does exactly that.
Cancellation
$/cancelRequest cancels a request that has been received and not started. The server queues what it receives and runs the queue when its input has nothing more buffered, so a cancellation that arrives with the request it names, or behind it in the same read (the usual case for a request waiting behind slow work), is applied first: the client gets RequestCancelled once, and the handler is never called. A handler that is already running is not interrupted — the server is single-threaded — so slow work should be kept in short calls or moved to a worker thread from a snapshot of the tree. docs/API.md states the rules.
Examples
Each example plays a scripted editor session in memory and prints what the server sent back. They share a complete language server for a tiny Lisp — grammar, analysis, and handler — in examples/common, written as a template for wiring in a real language.
| Example | Shows |
|---|---|
server |
Live diagnostics (syntax errors, plus undefined and duplicate names resolved with symbol-lang), hover, go-to-definition, completion, the outline, and semantic tokens. Run with -- --stdio to serve a real editor. |
encodings |
The same hover asked by a UTF-16 client and a UTF-8 client, and a column inside a surrogate pair snapping to its character. |
client_requests |
A server-to-client request (workspace/configuration) answered by the editor over in-memory channels, its answer through Handler::response, and republishing for every open document. |
Performance
Measured end to end through Server::run on in-memory streams, with only the measured messages timed (cargo bench; Windows 11, x86_64, release profile, one run). Documents are generated Lisp, about 34 bytes per line, with every tenth line a non-ASCII comment. The feature benchmarks run the example Lisp server over a 10,000-line document.
| Benchmark | What one iteration is | Time |
|---|---|---|
keystroke/1000 |
one didChange typing or deleting a character, 1,000-line document |
~2.8 µs |
keystroke/10000 |
the same, 10,000 lines (340 KB) | ~8.2 µs |
keystroke/100000 |
the same, 100,000 lines (3.4 MB) | ~116 µs |
request/hover |
a request round trip: queue, read, two position conversions, reply | ~1.1 µs |
publish/100 |
a keystroke whose handler publishes 100 diagnostics | ~22 µs |
publish/long_line_10000 |
a keystroke whose handler publishes 10,000 diagnostics, in scattered order, on one 39 KB non-ASCII line | ~1.9 ms |
open/10000 |
didOpen of a 340 KB document: JSON, line index, full parse, analysis not included |
~3.0 ms |
request/semantic_tokens |
semanticTokens/full for 10,000 lines (about 40,000 tokens): build, encode, write |
~5.8 ms |
request/document_symbols |
documentSymbol for 10,000 lines (about 9,000 symbols) |
~6.2 ms |
request/completion |
completion with about 9,000 items |
~1.9 ms |
There is no controlled comparison against 1.x on one machine in this table; the 1.x README quoted a different run. A keystroke's cost is dominated by incremental-lang's reparse and its text splice, which grows with the size of the file because the text is one contiguous string; lsp-lang's own share is a small fraction of it. What keeps it small:
- No value trees. The JSON-RPC envelope is decoded in place by
rpc-ioandjsonx-lib; parameters stay slices until read. Results are written straight to text: diagnostics, semantic tokens, outlines, and completion lists are never aValue. - One write per turn. Everything sent during one message or one queued job is written with a single
write_allandflush, and a turn that sends nothing costs no system call. Notifications the crate writes itself skip the transport's validate-and-copy (debug builds still validate every one). - An incremental line index. An edit drops and rescans only the line starts whose two deciding bytes it touched, then shifts the rest by a constant in one pass. Line breaks are found eight bytes at a time in plain safe Rust. Converting a position is a binary search over line starts plus, for UTF-16, a branch-free count over the line prefix (or nothing at all for an ASCII prefix, or in a UTF-8 session).
- Batched diagnostics. All the spans of a
publish_diagnosticscall are converted in one ascending pass, so ten thousand diagnostics on one long line cost one scan of it, not ten thousand.
Design notes
- Synchronous and single-threaded. The server decodes every complete message it holds, runs the work they queued in arrival order, and only then reads again. For an editor talking to one server over a pipe this is the lowest-latency design: no locks or executor stand between the client and the handler, and the gap between receiving a request and running it is what lets a cancellation arrive first. The cost is that a running handler cannot be interrupted. A handler with slow work can snapshot a document's tree (
O(1),Send + Sync) for a worker thread. - The server owns document sync. It advertises incremental sync with open/close notifications, applies every change, and falls back to a full parse when an incremental reparse fails. A change it cannot apply at all (a line the document lacks, a version that does not increase, a grammar that fails on the whole text) never loses the document: it keeps its last good state, is marked out of step, and comes back on the next whole-text change or reopen, and the client is told why with
window/logMessage. - Problems are reported, never panicked on. Malformed JSON and invalid requests are answered with JSON-RPC errors; malformed notifications and stray answers from the client, which cannot be answered, are logged to the client. A panic in the grammar or the handler is contained where panics unwind: a request whose handler panicked is answered with
InternalError, any other panic is logged, and the session carries on. Only a broken transport, broken framing, or the end of the session stopsrun. - Capabilities are a declaration.
Handler::featureslists what the handler offers, and the capabilities follow from it. Rust cannot tell which trait methods an implementation overrode, so the list is explicit: a feature listed without its method answersMethodNotFound, and a method overridden but not listed is never called. - Hand-written JSON, no derives. Every protocol type writes itself with straight-line code and reads itself with checked accessors that say what was wrong; there is no reflection and no macro.
- Types the protocol needs, no more. The five features and their results are typed; everything else is a
ValuethroughHandler::request, so the crate does not pin the server to one version of the full protocol type set.
Testing
Unit tests cover each module. Integration tests drive the server as an editor would: tests/server.rs through byte streams (the lifecycle, framing, every error response, document synchronization, diagnostics, server-to-client requests, panics), tests/conformance.rs as a conversation over in-memory pipes (messages split at every byte, pipelined requests, hang-ups), tests/features.rs for the typed features and cancellation, and tests/hostile.rs for malformed JSON, wrong types, huge and deeply nested messages, floods, and broken framing (nothing panics; every request is answered once). Property tests in tests/proptests.rs check, for arbitrary edit sequences in both encodings, that the server's text always equals the client's, that the tree always equals a fresh parse, that positions round-trip with surrogate pairs, that semantic tokens decode to a reference encoding, and that published diagnostics are well-formed. Every rust example in this README and in docs/API.md is compiled and run as a doctest.
Cross-platform support
The crate is pure Rust with no platform-specific code and no unsafe. It is tested on Linux, macOS, and Windows in CI, on stable and on the 1.85 MSRV.
Contributing
See REPS.md for the engineering standards every change is held to, and dev/ROADMAP.md for the roadmap. Before a PR: cargo fmt --all, cargo clippy --all-targets --all-features -- -D warnings, and cargo test --all-features must be clean.