Module tokenized 

Language adapter backed by the weavatrix-parse tokenizer.

The scanner this replaces reads lines. That is wrong in ways that change answers, and the failures are not exotic: a // inside a string ends the line early, a brace counted inside a comment moves every scope after it, a declaration written across three lines disappears, and every span it produces covers a whole line rather than the name it found.

Reading tokens fixes all of those at once, and brings languages the line scanner never had: HTML and CSS with the selector edge between them, Swift, Terraform, XML, the document formats, and shell scripts - where a CI job is often the only place a service endpoint is written down.

Structs

TokenizedAdapter

One language served by the tokenizer.