Expand description
Code fences and inline code spans: finding them, and blanking them so a lexer never harvests out of them.
One lexer for every caller. This crate’s link scanners, a #tag lexer and
any tool that rewrites markdown all ask “is this code”, and a second copy
would be a second answer.
CodeRules picks between two rule sets.
CodeRules::CommonMark, the default, follows CommonMark:
- A fence opens on a line of up to three spaces and a run of three or more backticks or tildes. A backtick fence’s info string may not hold a backtick. The fence closes on a line holding only a run of the same character, at least as long, and whitespace. An unclosed fence runs to the end of the input.
- An inline span opens on a run of backticks and closes on the next run of exactly that many. A run with no close is literal text.
- A span stays inside its block. A blank line, a fence, an ATX heading, a setext underline, a thematic break, a block quote or a list item ends the block. A span opened on an ATX heading stays on that line.
CodeRules::SkillExport keeps the looser rules of some agent-skill
renderers, for tools that must match them:
- A backtick fence’s info string may hold backticks.
- Any fence marker line of the same character and at least the same length closes a fence, whatever follows the run.
- A longer backtick run closes a shorter span.
- Only a blank line or a fence marker line ends a span’s block.
Both rule sets read one line at a time, so neither knows about container blocks. Indented code blocks are not code. A fence is found only when its marker starts within three spaces of the line start, so a fence nested in a list item or a block quote is missed. A list item or block quote marker ends a span’s block at any indent and with any start number, even with no text after it, so it may end a span that CommonMark would continue. Nested items are indented, and ending a span early keeps text visible.
CodeRules::CommonMark also treats a line starting with | after up to
three spaces as a one-line block, table or not. A span cannot enter or
leave that line, but a span in it may cross a cell. GFM reads a pipe line
without a delimiter row as paragraph text, and splits a table row into
cells before it reads spans.
Every marker is ASCII, so a byte scan never splits a UTF-8 character.
use markdown_link::code::{CodeRules, strip_code_with};
let text = "keep [[a]] `[[b]]`";
let blanked = strip_code_with(text, CodeRules::CommonMark);
assert_eq!(blanked, "keep [[a]] ");Structs§
- Fences
- The fence state of a line-by-line scan: which fence, if any, is open.
- Spans
- Finds where inline code spans close, for a scan that reads its input once, left to right.
Enums§
- Code
Rules - Which rules the lexer follows. See the module doc.
- Fence
Line - Where a line sits relative to fenced code, as
Fences::linereports it.
Functions§
- strip_
code - Blank out fenced and inline code so links and tags inside it are not
harvested.
strip_code_withunderCodeRules::CommonMark. - strip_
code_ with - Blank out fenced and inline code under
rules.