Crate textlane 

Textlane is a Rust-native text processing pipeline: a unified replacement whose plugins run natively, with Node.js and Python bindings over the same crate.

Re-exports

pub use textlane_html as html;

pub use textlane_md as md;

pub use textlane_mdx as mdx;

pub use textlane_nl as nl;

Modules

hast

The hast nodes: HTML’s syntax tree.

mdast

The mdast nodes: Markdown’s syntax tree.

nlcst

The nlcst nodes: natural language’s syntax tree.

Structs

Allocator

The arena every node of one document is allocated in. An arena to bump allocate into.

ArenaVec

A vector allocated in a document’s arena that never drops its items.

Attach

What plugins register while a processor freezes.

Error

A failure that stops a pipeline: a stable code and a message for people.

File

The document a pipeline processes: where it came from and what plugins reported about it.

Finished

A finished run: what it produced and the file plugins reported against.

FrozenProcessor

A processor whose plugins are attached: it runs any number of documents, in parallel.

HtmlTarget

What an HTML compiler writes that a stage writing HTML itself must match.

Lines

The line starts of a source, to turn byte points into unist’s UTF-16 lines and columns.

Location

A source indexed as vfile-location reads it: offsets in UTF-16 code units, turned into lines, columns and the byte points of core’s positions, each in constant or log time.

Message

A diagnostic a plugin reported, as vfile-message does.

NaturalLanguageParserSlot

The slot holding the natural-language parser bridges into nlcst read when their stage is built; the textlane processor seeds it with nl.parse’s Latin parser before plugins attach.

Output

What processing one document produced: the output and the kind of text it is.

Pause

A run stopped before consecutive host stages: the binding runs each of hosts over tree, in order, and hands the result to resume.

Pipeline

A processor built from plugin names and JSON options, with unified’s use, data and freeze semantics: what the bindings drive, and Rust code mixing its own plugins in.

Point

A place in the source: a byte offset, plus the virtual spaces into a tab it sits at.

ProcessJob

A frozen pipeline’s process, independent of the Pipeline it came from.

Processor

A pipeline being composed: .with() adds a plugin, as unified’s .use() does.

QuoteCharacters

The quote characters to use for double and single quotes.

ResolvedUrl

A relative or root-relative URL resolved to the file it names inside a site.

ScriptError

A syntax error: the byte span of what the parser stopped at, which ends at the end of the source when more source might still make it valid.

ScriptExpression

What an expression is, as far as MDX checks it: the byte offset where it ends, before any comments after it, and, for an object literal (parentheses dropped), the ESTree type and byte offset of each property.

ScriptParserSlot

The slot holding the script parser md.mdx validates JavaScript with; the textlane processor seeds it when built with mdx. Without one, expressions end at the brace that balances their opening one and imports and exports at a blank line.

ScriptStatement

A top-level statement of a module: its ESTree type, where it starts and, for an import, the local names it declares.

Slots

Data plugins share while attaching, one value per type.

Slugger

github-slugger’s BananaSlug: slugs, made unique by counting repeats.

TypographerSlot

The slot holding the typographer md.smartypants uses; the textlane processor seeds it when built with nl.

Typography

What a Typographer replaces, as retext-smartypants’ options name it.

UnistPoint

A point as unist writes it.

UnistPosition

A position as unist writes it: lines and columns from 1, offsets in UTF-16 code units.

Enums

Api

Whether a call may finish asynchronous stages, as unified’s process and run do, or must fail on them, as processSync and runSync do.

Backticks

How smartypants treats backticks: false, true or 'all'.

Dashes

How smartypants treats dashes: false, true, 'oldschool' or 'inverted'.

Data

unist’s data: whatever JSON a plugin attached to a node, kept as it is.

Ellipses

How smartypants treats dots: false, true, 'spaced' or 'unspaced'.

Outcome

What a run produced: the tree, from start_run, or the text, from start_process.

Position

Where a node sits: a span of the current source, or a unist position that does not point into it, such as one a foreign plugin set or one read from a serialized tree.

Progress

How far a run of a pipeline got: to its end, or to stages a binding must run.

Step

What starting a stage gives: its tree, or a job that yields it.

Tree

The tree a pipeline stage takes or returns.

TreeKind

Which kind of tree a stage takes or returns, checked when a processor freezes.

Traits

Compiler

Turns a tree into output text, whose kind extension names (html, md).

HtmlWriter

Writes HTML straight from a tree, or None to have the stages run as declared.

NaturalLanguageParser

A natural-language parser, as parse-latin’s instance is to mdast-util-to-nlcst and hast-util-to-nlcst: bridges into nlcst tokenize text with it and run its plugins.

Parser

Turns input text into a tree.

Plugin

A pipeline stage: attached once when a processor freezes, where it registers what it does.

ScriptParser

A JavaScript parser, as acorn is to micromark’s MDX extensions: md.mdx asks it where an expression or a block of imports and exports is valid, and keeps only the source.

Transform

Changes a tree, or bridges it into another kind of tree.

Typographer

Typographic punctuation for text, as retext-smartypants gives it: what md.smartypants asks the natural-language plugins for.

Functions

create_slug

Lowercase, drop punctuation and symbols, turn spaces into dashes.

find_gemoji_shortcodes

The :name: shortcodes gemoji knows in value, each (start, end, emoji), found as remark-gemoji’s /:(\+1|[-\w]+):/g finds them; an unknown name is retried from its next character, so its closing colon can open the next shortcode.

list_plugins

The name of every public plugin this build carries, as a pipeline names it.

prefix_base

base once before a root-relative URL: never twice, nothing for / or none.

read_gemoji_label

A shortcode’s name as remark-emoji labels it: each _, and each - not before 1, as a space.

read_messages

Read the messages a binding’s plugins reported, each {reason, ruleId?, source?, place?} as vfile-message has it, its place a point or a position.

read_retained_memory

The most memory, in bytes, each thread keeps after a document to reuse for the next.

read_switch

A switch upstream reads as !== false: null and a missing option mean true.

read_version

The crate’s version, which every binding reports as its own.

resolve_url

Resolve url, written in the file at path, to the file it names inside root; None for a URL with a scheme, protocol-relative, holding a backslash, a control character or surrounding space, or a fragment alone, or a relative one without a file to resolve it against or resolving outside root.

set_retained_memory

Cap the memory, in bytes, each thread keeps after a document to reuse for the next; 0 keeps none.

write_file

The file a binding’s process hands back, shaped as a vfile: {value, messages, data}, each message {reason, ruleId, source, place} in the order plugins reported them.

write_messages

Messages as a vfile holds them, each {reason, ruleId, source, place}, in order.

Type Aliases

ArenaString

A string allocated in a document’s arena, while it is being built.

Job

Deferred work a stage hands back, finished only by the asynchronous API.