Struct TextBuffer 

pub struct TextBuffer { /* private fields */ }

A text buffer that manages document content using a piece table with integrated line tracking

Implementations

impl TextBuffer

pub fn new( _large_file_threshold: usize, fs: Arc<dyn FileSystem + Send + Sync>, ) -> Self

Create a new text buffer with the given filesystem implementation. Note: large_file_threshold is ignored in the new implementation

pub fn new_with_path( large_file_threshold: usize, fs: Arc<dyn FileSystem + Send + Sync>, path: PathBuf, ) -> Self

Create an empty buffer associated with a file path. Used for files that don’t exist yet — the path is set so saving will create the file.

pub fn version(&self) -> u64

Current buffer version (monotonic, wraps on overflow)

pub fn filesystem(&self) -> &Arc<dyn FileSystem + Send + Sync>

Get a reference to the filesystem implementation used by this buffer.

pub fn set_filesystem(&mut self, fs: Arc<dyn FileSystem + Send + Sync>)

Set the filesystem implementation for this buffer.

pub fn from_bytes( content: Vec<u8>, fs: Arc<dyn FileSystem + Send + Sync>, ) -> Self

Create a text buffer from initial content with the given filesystem.

pub fn from_bytes_with_encoding( content: Vec<u8>, encoding: Encoding, fs: Arc<dyn FileSystem + Send + Sync>, ) -> Self

Create a text buffer from bytes with a specific encoding (no auto-detection).

pub fn from_str( s: &str, _large_file_threshold: usize, fs: Arc<dyn FileSystem + Send + Sync>, ) -> Self

Create a text buffer from a string with the given filesystem.

pub fn empty(fs: Arc<dyn FileSystem + Send + Sync>) -> Self

Create an empty text buffer with the given filesystem.

pub fn load_from_file<P: AsRef<Path>>( path: P, large_file_threshold: usize, fs: Arc<dyn FileSystem + Send + Sync>, ) -> Result<Self>

Load a text buffer from a file using the given filesystem.

pub fn load_from_file_with_encoding<P: AsRef<Path>>( path: P, encoding: Encoding, fs: Arc<dyn FileSystem + Send + Sync>, config: BufferConfig, ) -> Result<Self>

Load a text buffer from a file with a specific encoding (no auto-detection).

pub fn check_large_file_encoding( path: impl AsRef<Path>, fs: Arc<dyn FileSystem + Send + Sync>, ) -> Result<Option<LargeFileEncodingConfirmation>>

Check if loading a large file requires user confirmation due to encoding.

Some encodings (like Shift-JIS, GB18030, GBK, EUC-KR) cannot be “resynchronized” - meaning you cannot determine character boundaries when jumping into the middle of a file. These encodings require loading the entire file into memory.

Returns Some(confirmation) if user confirmation is needed, None if the file can be loaded with lazy/streaming loading.

pub fn load_large_file_confirmed( path: impl AsRef<Path>, fs: Arc<dyn FileSystem + Send + Sync>, ) -> Result<Self>

Load a large file, optionally forcing full load for non-resynchronizable encodings.

Called with force_full_load=true after user confirms the warning about non-resynchronizable encodings requiring full file loading.

pub fn save(&mut self) -> Result<()>

Save the buffer to its associated file

pub fn save_to_file<P: AsRef<Path>>(&mut self, path: P) -> Result<()>

Save the buffer to a specific file

Uses the write recipe approach for both local and remote filesystems:

• Copy ops reference unchanged regions in the source file
• Insert ops contain new/modified data

For remote filesystems, the recipe is sent to the agent which reconstructs the file server-side, avoiding transfer of unchanged content.

For local filesystems with ownership concerns (file owned by another user), uses in-place writing to preserve ownership. Otherwise uses atomic writes.

If the line ending format has been changed (via set_line_ending), all content will be converted to the new format during save.

pub fn finalize_external_save(&mut self, dest_path: PathBuf) -> Result<()>

Finalize buffer state after an external save operation (e.g., via sudo).

This updates the saved snapshot and file size to match the new state on disk.

pub fn total_bytes(&self) -> usize

Get the total number of bytes in the document

pub fn line_count(&self) -> Option<usize>

Get the total number of lines in the document Uses the piece tree’s integrated line tracking Returns None if line count is unknown (e.g., for large files without line indexing)

pub fn mark_saved_snapshot(&mut self)

Snapshot the current tree as the saved baseline

pub fn refresh_saved_root_if_unmodified(&mut self)

Refresh the saved root to match the current tree structure without clearing the modified flag. Call this after structural-only changes (e.g. chunk_split_and_load during search scan) so that diff_since_saved() can take the fast Arc::ptr_eq path.

pub fn diff_since_saved(&self) -> PieceTreeDiff

Diff the current piece tree against the last saved snapshot.

This compares actual byte content, not just tree structure. This means that if you delete text and then paste it back, the diff will correctly show no changes (even though the tree structure differs).

Uses a two-phase algorithm for efficiency:

• Phase 1: Fast structure-based diff to find changed byte ranges (O(num_leaves))
• Phase 2: Only compare actual content within changed ranges (O(edit_size))

This is O(edit_size) instead of O(file_size) for small edits in large files.

pub fn offset_to_position(&self, offset: usize) -> Option<Position>

Convert a byte offset to a line/column position

pub fn position_to_offset(&self, position: Position) -> usize

Convert a line/column position to a byte offset

pub fn insert_bytes(&mut self, offset: usize, text: Vec<u8>) -> Cursor

Insert text at the given byte offset

pub fn insert(&mut self, offset: usize, text: &str)

Insert text (from &str) at the given byte offset

pub fn insert_at_position( &mut self, position: Position, text: Vec<u8>, ) -> Cursor

Insert text at a line/column position This now uses the optimized piece_tree.insert_at_position() for a single traversal

pub fn delete_bytes(&mut self, offset: usize, bytes: usize)

Delete text starting at the given byte offset

pub fn delete(&mut self, range: Range<usize>)

Delete text in a range

pub fn delete_range(&mut self, start: Position, end: Position)

Delete text in a line/column range This now uses the optimized piece_tree.delete_position_range() for a single traversal

pub fn replace_content(&mut self, new_content: &str)

Replace the entire buffer content with new content This is an O(n) operation that rebuilds the piece tree in a single pass, avoiding the O(n²) complexity of applying individual edits.

This is used for bulk operations like “replace all” where applying individual edits would be prohibitively slow.

pub fn restore_buffer_state(&mut self, snapshot: &BufferSnapshot)

Restore a previously saved buffer state (for undo/redo of BulkEdit).

This restores the piece tree AND the buffers list, which is critical because consolidate_after_save() replaces self.buffers. Without restoring buffers, the piece tree would reference buffer IDs that no longer exist.

pub fn snapshot_buffer_state(&self) -> Arc<BufferSnapshot>

Snapshot the current buffer state (piece tree + buffers) for BulkEdit undo/redo.

The snapshot includes buffers because consolidate_after_save() can replace self.buffers between the snapshot and restore, which would otherwise cause the restored piece tree to reference nonexistent buffer IDs.

pub fn apply_bulk_edits(&mut self, edits: &[(usize, usize, &str)]) -> isize

Apply bulk edits efficiently in a single pass Returns the net change in bytes

pub fn get_text_range_mut( &mut self, offset: usize, bytes: usize, ) -> Result<Vec<u8>>

Get text from a byte offset range with lazy loading This will load unloaded chunks on-demand and always returns complete data

Returns an error if loading fails or if data cannot be read for any reason.

NOTE: Currently loads entire buffers on-demand. Future optimization would split large pieces and load only LOAD_CHUNK_SIZE chunks at a time.

pub fn prepare_viewport( &mut self, start_offset: usize, line_count: usize, ) -> Result<()>

Prepare a viewport for rendering

This is called before rendering with &mut access to pre-load all data that will be needed for the viewport. It estimates the number of bytes needed based on the line count and pre-loads them.

Arguments

• start_offset - The byte offset where the viewport starts
• line_count - The number of lines to prepare (estimate)

Returns

Ok(()) if preparation succeeded, Err if loading failed

pub fn to_string(&self) -> Option<String>

Get all text as a String Returns None if any buffers are unloaded (lazy loading)

pub fn len(&self) -> usize

Get the total number of bytes

pub fn is_empty(&self) -> bool

Check if the buffer is empty

pub fn file_path(&self) -> Option<&Path>

Get the file path associated with this buffer

pub fn rename_file_path(&mut self, path: PathBuf)

Update the file path after a rename operation on disk.

pub fn clear_file_path(&mut self)

Clear the file path (make buffer unnamed) Note: This does NOT affect Unloaded chunk file_paths used for lazy loading. Those still point to the original source file for chunk loading.

pub fn extend_streaming(&mut self, source_path: &Path, new_size: usize)

Extend buffer to include more bytes from a streaming source file. Used for stdin streaming where the temp file grows over time. Appends a new Unloaded chunk for the new bytes.

pub fn is_modified(&self) -> bool

Check if the buffer has been modified since last save

pub fn clear_modified(&mut self)

Clear the modified flag (after save)

pub fn set_modified(&mut self, modified: bool)

Set the modified flag explicitly Used by undo/redo to restore the correct modified state

pub fn is_recovery_pending(&self) -> bool

Check if buffer has pending changes for recovery auto-save

pub fn set_recovery_pending(&mut self, pending: bool)

Mark buffer as needing recovery auto-save (call after edits)

pub fn is_large_file(&self) -> bool

Check if this is a large file with lazy loading enabled

pub fn has_line_feed_scan(&self) -> bool

Check if line feeds have been scanned for this large file. When true, line_count() returns exact values.

pub fn piece_tree_leaves(&self) -> Vec<LeafData>

Get the raw piece tree leaves (for storing alongside scan chunks).

pub fn prepare_line_scan(&mut self) -> (Vec<LineScanChunk>, usize)

Prepare work items for an incremental line scan.

First splits any oversized leaves in the piece tree so every leaf is at most LOAD_CHUNK_SIZE bytes. Then returns one work item per leaf. After scanning, get_text_range_mut will never need to split a scanned leaf (it’s already chunk-sized), so line-feed counts are preserved.

Returns (chunks, total_bytes).

pub fn search_scan_init( &mut self, regex: Regex, max_matches: usize, query_len: usize, ) -> ChunkedSearchState

Initialize a chunked search scan over this buffer’s piece tree.

Used for in-editor Ctrl+F (incremental, yields to the event loop between chunks) and for searching dirty buffers during project grep. For searching files on disk, use FileSystem::search_file instead.

pub fn search_scan_next_chunk( &mut self, state: &mut ChunkedSearchState, ) -> Result<bool>

Process one chunk of a chunked search scan.

Loads the next chunk via get_text_range_mut, prepends overlap from the previous chunk, runs the regex, and appends matches to state with line/column/context computed on the fly from the loaded bytes.

Line numbers are tracked incrementally via running_line — each chunk counts newlines in its non-overlap portion to advance the counter for the next chunk, and matches use an incremental cursor so total line-counting work is O(chunk_size), not O(chunk × matches).

Returns Ok(true) if there are more chunks to process, Ok(false) when the scan is complete.

TODO: For concurrent/parallel search (searching multiple files at once), chunks would need to return chunk-relative line numbers and have them fixed up with each file’s starting line offset after all chunks complete.

pub fn search_scan_all( &mut self, regex: Regex, max_matches: usize, query_len: usize, ) -> Result<ChunkedSearchState>

Run a complete chunked search over the piece tree (all chunks).

Synchronous variant — used for dirty buffer snapshots in project grep and in tests. For on-disk files, use FileSystem::search_file.

pub fn search_hybrid_plan(&mut self) -> Option<HybridSearchPlan>

Build a hybrid search plan from the piece tree.

Extracts regions (unloaded file ranges + loaded in-memory data) that can be searched independently. The plan is Send so it can be executed on a background thread via HybridSearchPlan::execute.

Returns None if the buffer has no file path (caller should fall back to search_scan_all).

pub fn search_hybrid( &mut self, pattern: &str, opts: &FileSearchOptions, regex: Regex, max_matches: usize, query_len: usize, ) -> Result<Vec<SearchMatch>>

Hybrid search: uses fs.search_file for unloaded piece-tree regions (searches where the data lives, no network transfer) and in-memory regex for loaded/edited regions. Handles overlap at region boundaries.

For a huge remote file with a small local edit, this avoids transferring the entire file — only match metadata crosses the network.

Falls back to search_scan_all when the buffer has no file path or is fully loaded.

pub fn scan_leaf(&self, leaf: &LeafData) -> Result<usize>

Count \n bytes in a single leaf.

Uses count_line_feeds_in_range for unloaded buffers, which remote filesystem implementations can override to count server-side.

pub fn leaf_io_params(&self, leaf: &LeafData) -> Option<(PathBuf, u64, usize)>

Return the I/O parameters for an unloaded leaf, or None if loaded.

Used by the incremental scan to distinguish leaves that can be counted in-memory (via scan_leaf) from those that need filesystem I/O.

pub fn buffer_slice(&self) -> &[StringBuffer]

Get a reference to the string buffers (for parallel scanning).

pub fn apply_scan_updates(&mut self, updates: &[(usize, usize)])

Apply the results of an incremental line scan.

pub fn rebuild_with_pristine_saved_root( &mut self, scan_updates: &[(usize, usize)], )

After an incremental line-feed scan completes, rebuild the tree so that saved_root and the current tree share Arc pointers for unedited subtrees. This makes diff_since_saved() O(edited regions) instead of O(file size).

pub fn resolve_line_byte_offset(&mut self, target_line: usize) -> Option<usize>

Resolve the exact byte offset for a given line number (0-indexed).

Uses the tree’s line feed counts to find the piece containing the target line, then loads/reads that piece’s data to find the exact newline position. This works even when buffers are unloaded (large file with scanned line index).

pub fn original_file_size(&self) -> Option<usize>

Get the saved file size (size of the file on disk after last load/save) For large files, this is used during recovery to know the expected original file size. Returns None for new unsaved buffers.

pub fn get_recovery_chunks(&self) -> Vec<(usize, Vec<u8>)>

Get recovery chunks for this buffer (only modified portions)

For large files, this returns only the pieces that come from Added buffers (i.e., the modifications), not the original file content. This allows efficient incremental recovery without reading/writing the entire file.

Returns: Vec of (original_file_offset, data) for each modified chunk The offset is the position in the ORIGINAL file where this chunk should be inserted.

pub fn is_binary(&self) -> bool

Check if this buffer contains binary content

pub fn line_ending(&self) -> LineEnding

Get the line ending format for this buffer

pub fn set_line_ending(&mut self, line_ending: LineEnding)

Set the line ending format for this buffer

This marks the buffer as modified since the line ending format has changed. On save, the buffer content will be converted to the new format.

pub fn set_default_line_ending(&mut self, line_ending: LineEnding)

Set the default line ending format for a new/empty buffer

Unlike set_line_ending, this does NOT mark the buffer as modified. This should be used when initializing a new buffer with a configured default.

pub fn encoding(&self) -> Encoding

Get the encoding format for this buffer

pub fn set_encoding(&mut self, encoding: Encoding)

Set the encoding format for this buffer

This marks the buffer as modified since the encoding format has changed. On save, the buffer content will be converted to the new encoding.

pub fn set_default_encoding(&mut self, encoding: Encoding)

Set the default encoding format for a new/empty buffer

Unlike set_encoding, this does NOT mark the buffer as modified. This should be used when initializing a new buffer with a configured default.

pub fn detect_line_ending(bytes: &[u8]) -> LineEnding

Detect the line ending format from a sample of bytes

Uses majority voting: counts CRLF, LF-only, and CR-only occurrences and returns the most common format.

pub fn detect_encoding(bytes: &[u8]) -> Encoding

Detect the text encoding from a sample of bytes

Delegates to the encoding module. Use detect_encoding_or_binary when you need to know if the content should be treated as binary.

pub fn detect_encoding_or_binary( bytes: &[u8], truncated: bool, ) -> (Encoding, bool)

Detect the text encoding and whether content is binary.

Returns (Encoding, is_binary) where:

• Encoding is the detected encoding (or default if binary)
• is_binary is true if the content should be treated as raw binary

Delegates to the encoding module for detection logic.

pub fn detect_and_convert_encoding(bytes: &[u8]) -> (Encoding, Vec<u8>)

Detect encoding and convert bytes to UTF-8

Returns the detected encoding and the UTF-8 converted content. This is the core function for normalizing file content to UTF-8 on load.

pub fn convert_to_encoding( utf8_bytes: &[u8], target_encoding: Encoding, ) -> Vec<u8>

Convert UTF-8 content to the specified encoding for saving

Used when saving files to convert internal UTF-8 representation back to the original (or user-selected) encoding. Note: This does NOT add BOM - the BOM is handled separately in build_write_recipe.

pub fn normalize_line_endings(bytes: Vec<u8>) -> Vec<u8>

Normalize line endings in the given bytes to LF only

Converts CRLF (\r\n) and CR (\r) to LF (\n) for internal representation. This makes editing and cursor movement simpler while preserving the original format for saving.

pub fn get_line(&self, line: usize) -> Option<Vec<u8>>

Get text for a specific line

pub fn line_start_offset(&self, line: usize) -> Option<usize>

Get the byte offset where a line starts

pub fn piece_info_at_offset(&self, offset: usize) -> Option<PieceInfo>

Get piece information at a byte offset

pub fn stats(&self) -> TreeStats

Get tree statistics for debugging

pub fn find_next(&self, pattern: &str, start_pos: usize) -> Option<usize>

Find the next occurrence of a pattern, with wrap-around

pub fn find_next_in_range( &self, pattern: &str, start_pos: usize, range: Option<Range<usize>>, ) -> Option<usize>

Find the next occurrence of a pattern within an optional range If range is None, searches the entire buffer with wrap-around (same as find_next) If range is Some, searches only within that range without wrap-around

pub fn find_next_regex(&self, regex: &Regex, start_pos: usize) -> Option<usize>

Find the next occurrence of a regex pattern, with wrap-around

pub fn find_next_regex_in_range( &self, regex: &Regex, start_pos: usize, range: Option<Range<usize>>, ) -> Option<usize>

Find the next occurrence of a regex pattern within an optional range

pub fn replace_range(&mut self, range: Range<usize>, replacement: &str) -> bool

Replace a range with replacement text

pub fn replace_next( &mut self, pattern: &str, replacement: &str, start_pos: usize, range: Option<Range<usize>>, ) -> Option<usize>

Find and replace the next occurrence of a pattern

pub fn replace_all(&mut self, pattern: &str, replacement: &str) -> usize

Replace all occurrences of a pattern with replacement text

pub fn replace_all_regex( &mut self, regex: &Regex, replacement: &str, ) -> Result<usize>

Replace all occurrences of a regex pattern with replacement text

pub fn position_to_line_col(&self, byte_pos: usize) -> (usize, usize)

Convert byte position to (line, column) in bytes

pub fn line_col_to_position(&self, line: usize, character: usize) -> usize

Convert (line, character) to byte position - 0-indexed character is in BYTES, not UTF-16 code units Optimized to use single line_range() call instead of two

pub fn position_to_lsp_position(&self, byte_pos: usize) -> (usize, usize)

Convert byte position to LSP position (line, UTF-16 code units) LSP protocol uses UTF-16 code units for character offsets

pub fn lsp_position_to_byte(&self, line: usize, utf16_offset: usize) -> usize

Convert LSP position (line, UTF-16 code units) to byte position LSP uses UTF-16 code units for character offsets, not bytes Optimized to use single line_range() call instead of two

pub fn prev_char_boundary(&self, pos: usize) -> usize

Find the previous character boundary (UTF-8 aware)

pub fn next_char_boundary(&self, pos: usize) -> usize

Find the next character boundary (UTF-8 aware)

pub fn snap_to_char_boundary(&self, pos: usize) -> usize

Snap position to a valid UTF-8 character boundary If already at a boundary, returns the same position. Otherwise, moves to the previous valid boundary.

pub fn prev_grapheme_boundary(&self, pos: usize) -> usize

Find the previous grapheme cluster boundary (for proper cursor movement with combining characters)

This handles complex scripts like Thai where multiple Unicode code points form a single visual character (grapheme cluster). For example, Thai “ที่” is 3 code points but 1 grapheme cluster.

pub fn next_grapheme_boundary(&self, pos: usize) -> usize

Find the next grapheme cluster boundary (for proper cursor movement with combining characters)

This handles complex scripts like Thai where multiple Unicode code points form a single visual character (grapheme cluster). For example, Thai “ที่” is 3 code points but 1 grapheme cluster.

pub fn prev_word_boundary(&self, pos: usize) -> usize

Find the previous word boundary

pub fn next_word_boundary(&self, pos: usize) -> usize

Find the next word boundary

pub fn line_iterator( &mut self, byte_pos: usize, estimated_line_length: usize, ) -> LineIterator<'_>

Create a line iterator starting at the given byte position

This iterator lazily loads chunks as needed, never scanning the entire file. For large files with unloaded buffers, chunks are loaded on-demand (1MB at a time).

pub fn iter_lines_from( &mut self, byte_pos: usize, max_lines: usize, ) -> Result<TextBufferLineIterator>

Iterate over lines starting from a given byte offset, with line numbers

This is a more efficient alternative to using line_iterator() + offset_to_position() because it calculates line numbers incrementally during iteration by accumulating line_feed_cnt from pieces (which is already tracked in the piece tree).

Returns: Iterator yielding (byte_offset, content, line_number: Option)

• line_number is Some(n) for small files with line metadata
• line_number is None for large files without line metadata

Performance

• O(1) per line for line number calculation (vs O(log n) per line with offset_to_position)
• Uses single source of truth: piece tree’s existing line_feed_cnt metadata

pub fn get_line_number(&self, byte_offset: usize) -> usize

Get the line number for a given byte offset

Returns exact line number if metadata available, otherwise estimates based on bytes.

Behavior by File Size:

• Small files (< 1MB): Returns exact line number from piece tree’s line_starts metadata
• Large files (≥ 1MB): Returns estimated line number using byte_offset / estimated_line_length

Large files don’t maintain line metadata for performance reasons. The estimation uses the configured estimated_line_length (default 80 bytes).

pub fn estimated_line_length(&self) -> usize

Get the configured estimated line length for approximate line number calculations.

pub fn populate_line_cache( &mut self, start_byte: usize, _line_count: usize, ) -> usize

Get the starting line number at a byte offset (used for viewport rendering)

Line Cache Architecture (Post-Refactoring):

The concept of a separate “line cache” is now obsolete. After the refactoring, line tracking is integrated directly into the piece tree via:

BufferData::Loaded {
    data: Vec<u8>,
    line_starts: Option<Vec<usize>>  // None = large file mode (no line metadata)
}

Why This Method Still Exists:

The rendering code needs to know what line number to display in the margin at the top of the viewport. This method returns that line number, handling both small and large file modes transparently.

Small vs Large File Modes:

• Small files: line_starts = Some(vec) → returns exact line number from metadata
• Large files: line_starts = None → returns estimated line number (byte_offset / estimated_line_length)

Legacy Line Cache Methods:

These methods are now no-ops and can be removed in a future cleanup:

• invalidate_line_cache_from() - No-op (piece tree updates automatically)
• handle_line_cache_insertion() - No-op (piece tree updates automatically)
• handle_line_cache_deletion() - No-op (piece tree updates automatically)
• clear_line_cache() - No-op (can’t clear piece tree metadata)

Bug Fix (2025-11):

Previously this method always returned 0, causing line numbers in the margin to always show 1, 2, 3… regardless of scroll position. Now it correctly returns the actual line number at start_byte.

pub fn get_cached_byte_offset_for_line( &self, line_number: usize, ) -> Option<usize>

Get cached byte offset for line (compatibility method)

pub fn invalidate_line_cache_from(&mut self, _byte_offset: usize)

Invalidate line cache from offset (no-op in new implementation)

pub fn handle_line_cache_insertion( &mut self, _byte_offset: usize, _bytes_inserted: usize, )

Handle line cache insertion (no-op in new implementation)

pub fn handle_line_cache_deletion( &mut self, _byte_offset: usize, _bytes_deleted: usize, )

Handle line cache deletion (no-op in new implementation)

pub fn clear_line_cache(&mut self)

Clear line cache (no-op in new implementation)