Crate lsp_document

Helpers to convert between LSP representations of text documents and Rust strings.

Motivation:

LSP uses UTF16-encoded strings while Rust’s strings are UTF8-encoded. This means that text offsets in LSP and in Rust are different:

• LSP offsets are in 16-bit code-units and each character is either 1 or 2 of those,
• Rust strings are indexed in bytes and each character takes from 1 to 4 bytes.

To ensure that LSP client and server “talk” about the same part of a text document we need a translation layer.

Structure

There are two traits that define the basic functionality on text documents:

• TextMap defines operations to convert between byte offsets and Pos’s inside a UTF8-encoded string.
• TextAdapter defines operations to convert between LSP positions, native positions, and derived types.

The work-horse struct that implements both of these traits is IndexedText. It wraps the original text and can act as a replacement for String where relevant. The struct is generic in the type of text it wraps, however, so depending on the use-case it can be either:

• IndexedText<&str> when you don’t really need an ownership of the original text, or
• IndexedText<Arc<str>> otherwise.

Example usage

Below is a an example where the original text is &'static str.

use lsp_document::{TextMap, TextAdapter, Pos, IndexedText};
use lsp_types::Position;

// Character width
// U16:     1111111111111 1111111111 1 11 1 1 111111111 21
// U8:      1111111111111 1222122221 1 13 3 3 111111111 41
// U8 offset
//          0         1       2      3       4          5
//          0123456789012 3468013579 0 12 5 8 123456789 04
let text = "Hello, world!\nКак дела?\r\n做得好\nThis is 💣!";
let text = IndexedText::new(text);
//
// Examples of using TextMap methods
//
// Pos of 💣 from its offset
assert_eq!(text.offset_to_pos(50).unwrap(), Pos::new(3, 8));
// Raw line range info
assert_eq!(text.line_range(2).unwrap(), Pos::new(2, 0)..Pos::new(2, 10));
// Extracting part of text between two positions
assert_eq!(text.substr(Pos::new(1, 7)..Pos::new(1, 15)).unwrap(), "дела");

//
// Example of using TextAdapter methods
//
// Pos of `!` after 💣
assert_eq!(text.lsp_pos_to_pos(&Position::new(3, 10)).unwrap(), Pos::new(3, 12));
assert_eq!(text.pos_to_lsp_pos(&Pos::new(3, 12)).unwrap(), Position::new(3, 10));

Structs

IndexedText

A combo of TextMap + TextAdapter. Wraps the original text and provides all the conversion methods.

Pos

Native position inside a text document/string. Points to a valid position before the character inside a UTF8-encoded string.

TextChange

Native representation of a change that replaces a part of the target text.

Traits

TextAdapter

Defines operations to convert between native text types and lsp_types. The trait is automatically derived for any type that implements TextMap.

TextMap

Defines operations to convert between byte offsets and native Pos.

Functions

apply_change

Applies a TextChange to IndexedText returning a new text as String.