Skip to main content

djot_to_plain_text

Function djot_to_plain_text 

Source
pub fn djot_to_plain_text(djot: &str, options: &DjotImportOptions) -> String
Expand description

The prose of a Djot document, with no entities, no store, and no threads.

parse_djot and ParsedElement::flatten_to_blocks were both already pub; nothing chained them. This does, and that is the whole trick: it stops at the parse, where a full import goes on to create a Block entity per paragraph, list item and table cell, mirror each into the rope, and write its format runs.

§Why a project-wide search needs this

A host app searching a manuscript must ask “does this scene contain that word” of thousands of Djot rows, on every keystroke. Doing that by importing each one into a document is not a slow feature, it is a frozen app.

And searching the Djot source instead — the tempting shortcut — is simply wrong: the source is markup. http matches inside a link’s URL, * matches an emphasis marker, and an occurrence count taken from the source does not agree with what a replace re-derives inside the parsed document. Where a replace guards itself with “the text moved under me, skip this field”, a count taken from markup makes that guard fire on perfectly good rows.

§The contract

The result is byte-identical to the text the document searches for the same Djot: each block’s spans concatenated, blocks joined by a single \n, and a table announced by its TABLE_ANCHOR sentinel — exactly the string the import mirrors into the rope and that build_full_text_via_store recomposes. So an offset found here is an offset the document agrees with.

A property in djot_roundtrip_tests pins that across the whole generated feature set; without it this would be a second, silently-diverging definition of “the text”.

⚠ It is not the same as TextDocument::to_plain_text(), which walks frames and therefore orders a blockquote’s prose differently ("> a0\n\na" exports as "a\na0" but is searched as "a0\na"). The authority is what a search sees, because that is what a replace edits. See claude_reviews/text-document-plain-text-ordering.md.