Module processing 

Text utilities for markdown parsing and chunking.

Line splitting, fence/heading detection, token estimation, wikilink parsing, and keyword/path normalization. Ported from the TypeScript Talon implementation.

Structs

LineSpan

A line span within the original content.

ParsedWikiLink

Parsed components of a wikilink.

Constants

TOKEN_CHAR_RATIO

Token-to-character ratio for rough token estimation.

Functions

estimate_tokens

Estimates the number of tokens in text using a character ratio.

is_fence_line

Checks if a line is a fenced code block (3+ backticks or tildes).

is_heading_line

Checks if a line is an ATX heading (1-6 hash characters followed by space).

normalize_keyword

Normalizes a keyword for comparison: NFD normalization + lowercase + trim.

normalize_vault_path

Normalizes a vault path: backslashes to forward slashes, NFD normalization.

parse_wikilink

Parses a raw wikilink string into components.

split_lines

Splits markdown content into line spans.

strip_heading_text

Strips heading markers from a heading line.

strip_outer_quotes

Strips outer matching quotes from a string.