# MD084 - Invisible and discouraged Unicode characters
Aliases: `invisible-characters`
**Disabled by default.** This rule is opt-in: enable it explicitly with
`extend-enable`.
## What this rule does
This rule detects:
- invisible Unicode characters that can cause copy/paste and rendering bugs,
confuse readers, or block `rumdl` from applying auto-fixes.
Those characters are typically introduced in your documents by copy/pasting contents from the Web,
and are by nature very difficult to spot, until they create issues later on.
- characters Unicode itself steers authors away from: the code points carrying the
`Deprecated` property, and the ones UTR#20 lists as unsuitable for use with markup.
By default, to reduce the number of false positives, it reports only suspicious patterns:
- multiple consecutive invisible characters
- any invisible character at the start or end of a line
- invisible characters adjacent to any whitespace character
- deprecated and markup-unsuitable code points, anywhere they appear
It doesn't report any invisible character surrounded on both sides by visible characters,
since that is how those invisible characters are typically used, so it might miss some findings.
It also doesn't report two kinds of character that are presentation rather than hidden
content, because each is part of a glyph the reader actually sees:
- a variation selector that follows a base character. `U+26A0 U+FE0F` renders as the
emoji `⚠️`, while `U+26A0` on its own renders as the text glyph `⚠`.
- a zero width joiner that fuses the characters on either side of it, as in the
rainbow flag `🏳️🌈` (`U+1F3F3 U+FE0F U+200D U+1F308`) or `👩❤️👨`. Removing the
joiner would split the emoji into separate glyphs.
Either one is still reported when it isn't doing that job: at the start or end of a
line, next to whitespace, or with a character that draws no glyph where its base
should be. A duplicated variation selector is reported, since only the first one has
a base to modify, and so is one that follows an interlinear annotation delimiter.
Line endings are never reported or removed. CommonMark counts a lone carriage return
as a line ending in its own right, so a document written with classic-Mac line endings
is a single line as far as this rule is concerned, and every carriage return in it sits
at a line boundary or beside whitespace. Removing one would join two lines, or drop the
document's final line ending.
Presentation characters, the interlinear annotation delimiters and line endings are
never reported or removed by those three triggers, but they draw no glyph either, so
they still count when looking for consecutive ones and nothing can hide behind them.
The zero-width space in `⚠️x` is reported, and the auto-fix removes only the
zero-width space.
With `strict = true`, it reports any detected invisible character anywhere, other than
the line endings. Strict mode widens which hidden characters are worth reporting, which
is a judgment the allow-list already lets you make; it is not a licence to restructure
the document.
The list of non-visible characters that this rule will trigger on is:
| U+0000 to U+001F | C0 control characters, except tab (U+0009) and the line endings U+000A and U+000D |
| U+007F to U+009F | DEL + C1 control characters |
| U+00AD | SOFT HYPHEN |
| U+034F | COMBINING GRAPHEME JOINER |
| U+061C | ARABIC LETTER MARK |
| U+115F | HANGUL CHOSEONG FILLER |
| U+1160 | HANGUL JUNGSEONG FILLER |
| U+17B4 | KHMER VOWEL INHERENT AQ |
| U+17B5 | KHMER VOWEL INHERENT AA |
| U+180B to U+180E | Mongolian variation selectors + MONGOLIAN VOWEL SEPARATOR |
| U+200B to U+200F | ZWSP, ZWNJ, ZWJ, LRM, RLM |
| U+202A to U+202E | Bidi embedding/override controls |
| U+2060 to U+206F | WORD JOINER, invisibles, and bidi isolate controls |
| U+3164 | HANGUL FILLER |
| U+FE00 to U+FE0F | Variation Selectors (VS1..VS16) |
| U+FEFF | ZERO WIDTH NO-BREAK SPACE (BOM) |
| U+FFA0 | HALFWIDTH HANGUL FILLER |
| U+FFF0 to U+FFF8 | Reserved non-rendering specials |
| U+1BCA0 to U+1BCA3 | Shorthand format controls |
| U+1D173 to U+1D17A | Musical symbol format controls |
| U+E0000 to U+E0FFF | Tags block + Variation Selectors Supplement |
The deprecated Unicode characters, meaning every code point carrying the `Deprecated`
property. They still render, so the rule reports them without a fix: only the author
knows what the text should say instead.
| U+0149 | LATIN SMALL LETTER N PRECEDED BY APOSTROPHE |
| U+0673 | ARABIC LETTER ALEF WITH WAVY HAMZA ABOVE |
| U+0F77 | TIBETAN VOWEL SIGN VOCALIC LL |
| U+0F79 | TIBETAN VOWEL SIGN VOCALIC LR |
| U+17A3 | KHMER INHERENT VOWEL SIGN AA |
| U+17A4 | KHMER INHERENT VOWEL SIGN AE |
| U+206A to U+206F | INHIBIT SYMMETRIC SWAPPING to NOMINAL DIGIT SHAPES |
| U+2329 | LEFT-POINTING ANGLE BRACKET |
| U+232A | RIGHT-POINTING ANGLE BRACKET |
| U+E0001 | LANGUAGE TAG |
The characters [UTR#20][utr20] lists as unsuitable for use with markup that aren't
already covered above. These are visible or structural rather than hidden, so removing
one would drop content or leave a paired construct half-open. Only the two tone marks
have a replacement that preserves the text exactly, and only those are auto-fixed.
| U+0340 | COMBINING GRAVE TONE MARK | U+0300 |
| U+0341 | COMBINING ACUTE TONE MARK | U+0301 |
| U+FFF9 to U+FFFB | Interlinear annotation anchor, separator and terminator | |
| U+FFFC | OBJECT REPLACEMENT CHARACTER | |
U+206A to U+206F appear in both this rule's invisible list and its deprecated list.
Where one of the invisible triggers already reports such a character, that finding
wins, because it comes with a removal fix the deprecated finding cannot offer. Every
character is reported at most once.
References:
- [Unicode characters with `Deprecated=Yes`][deprecated]
- [UTR#20: Unicode in XML and other Markup Languages][utr20]
[deprecated]: https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=[:Deprecated=Yes:]
[utr20]: https://www.unicode.org/reports/tr20/tr20-9.html#Suitable
## Why this matters
Invisible code points are difficult to spot in reviews and can produce subtle
behavior differences across editors, terminals, and renderers. Catching them
early keeps Markdown content predictable and easier to maintain.
## Examples
### Incorrect (default mode)
```text
AB
```
```text
Starts with hidden char
```
```text
left right
```
### Correct
```text
AB
```
```text
Starts with visible text
```
```text
left right
```
## Configuration
```toml
[MD084]
strict = false
allow = []
```
- `strict`: When `true`, report any invisible character occurrence. The deprecated
and markup-unsuitable code points are reported in both modes.
Default: `false`.
- `allow`: Allow-list of codepoints to ignore, written as `U+XXXX`,
`U+XXXXX`, or `U+XXXXXX` (for example `U+200B`). Default: `[]`.
### Strict mode
```toml
[MD084]
strict = true
```
Strict mode is deliberately literal, so it also reports variation selectors that
default mode treats as presentation. On content using emoji such as `⚠️` or `❤️`
that means a finding per emoji, and the auto-fix would strip the selector and change
the rendered glyph. Allow-list the selector to keep those intact:
```toml
[MD084]
strict = true
allow = ["U+FE0F"]
```
### Allow-list example
```toml
[MD084]
allow = ["U+200B", "U+2060"]
```
Allow-list entries must use Unicode codepoint format. Invalid entries are
rejected during config parsing.
## Notes
- The rule checks all content, including code blocks and frontmatter.