Expand description
Reference specification of OpenJDK’s java.util.regex.Pattern behavior —
the rules this crate reproduces.
§OpenJDK java.util.regex Reference Specification
This document specifies what OpenJDK 25’s java.util.regex.Pattern actually does — not what the Javadoc says. The Javadoc is the official spec, but it is underspecified in dozens of places and, in several others, OpenJDK’s parser/engine diverges from a literal reading. Because OpenJDK is by far the dominant JVM implementation (Oracle JDK, GraalVM, Amazon Corretto, Azul Zulu, IBM Semeru / OpenJ9, etc.), “what OpenJDK does” is the working definition of Java regex.
This spec describes that working definition. Wherever it differs from a naive reading of the Javadoc, the divergence is called out and cross-referenced to QUIRKS.md.
Scope. Java 8+, targeting Java 13+ semantics (OpenJDK 25). Excludes the CANON_EQ (canonical equivalence) flag, which is rarely used and not implemented by most third-party engines.
Conventions. (pattern) in headings denotes the regex construct; → Some(result) denotes the matcher’s expected output; [Q-N] cross-references quirk #N in QUIRKS.md.
§Table of contents
- 1. Pattern syntax
- 2. Flags and inline flag scoping
- 3. Matching operations
- 4. Capture-group state model
- 5.
split() - 6. Replacement DSL (
appendReplacement/replaceAll) - 7. Compile-time errors
- 8. Quirks index
§1. Pattern syntax
A pattern is a string of characters interpreted by the regex compiler. Characters fall into three categories:
- Literal characters — match themselves (e.g.,
amatchesa). - Metacharacters — have special meaning:
\ . * + ? ( ) [ ] { } | ^ $ - = ! < > /(the last three only in specific contexts). - Escape sequences — backslash + character, with specific meanings (see §1.1).
Whitespace in the pattern is treated literally unless the COMMENTS flag ((?x) or Pattern.COMMENTS) is active, in which case unescaped whitespace and # … end-of-line comments are stripped (see §2). COMMENTS-mode whitespace stripping applies both outside and inside [...] character classes.
§1.1 Literals and escape sequences
A backslash escapes the next character. Every character following \ has one of the meanings below; an unrecognized character produces an “Illegal/unsupported escape sequence” error.
| Escape | Meaning |
|---|---|
\\ | Literal backslash |
\. \* \+ \? \( \) \[ \] \{ \} | \^ \$ | The corresponding metacharacter as a literal |
\- \! \= \< \> \/ \# \ \& \~ \@ \`` ' " , ; :` | These ASCII characters as literals (no-op escapes — Java permits backslash before any of them) |
\n | Newline (U+000A) |
\r | Carriage return (U+000D) |
\t | Horizontal tab (U+0009) |
\f | Form feed (U+000C) |
\a | Bell (U+0007) |
\e | Escape (U+001B) |
\0n | Octal value n (1–3 octal digits) |
\0nn | Octal value nn |
\0mnn | Octal value mnn where 0 ≤ m ≤ 3, 0 ≤ n ≤ 7 |
\xhh | Hex value hh (exactly 2 hex digits) |
\uhhhh | Hex value hhhh (exactly 4 hex digits, U+0000..U+FFFF) |
\x{hh..h} | Hex value with any number of digits, U+0000..U+10FFFF (supplementary chars allowed) |
\cX | Control character: the character corresponding to X XOR 0x40 (e.g., \cM = U+000D, \c@ = U+0000) |
\R | Any Unicode line break — see §1.6 |
\X | A single grapheme cluster (UAX #29 extended grapheme cluster, including ZWJ sequences) |
\Q…\E | Literal block: every character between \Q and \E matches literally, with no regex interpretation (see below) |
\p{…}, \P{…} | Unicode property / negated Unicode property — see §1.9 |
\d \D \w \W \s \S \h \H \v \V | Predefined character classes — see §1.2 |
\b \B \A \z \Z \G | Anchors — see §1.6 |
\1 … \9 (or longer) | Numbered backreference — see §1.8 |
\k<name> | Named backreference — see §1.8 |
\N{…} | (Reserved syntax — not supported by OpenJDK’s Pattern; it errors.) |
Octal escape rule. OpenJDK requires the leading 0 for octal: \0377 is octal 0xFF, but \377 is the backreference \3 followed by literal 7, 7. (Multiple regex engines accept \nnn as octal even without the leading 0; Java does not.)
\Q…\E semantics.
- Inside
\Q…\E, every character — including\,.,*,(, etc. — matches literally. No escape sequences are processed. - An unclosed
\Q(no\Ebefore end-of-pattern) consumes characters to the end of the pattern, equivalent to “literal rest of pattern.” \Q\E(empty literal block) emits nothing and is a no-op (it does not match any character). When at the start of an alternation branch followed by|, it acts as the empty branch.- A quantifier following
\Q…\Equantifies the last character of the block —\Qabc\E+matchesabfollowed by one or morecs, not one or moreabcs. To quantify the whole block, wrap it in a group:(?:\Qabc\E)+. \Q…\Emay appear inside a character class:[\Qabc\E]is equivalent to[abc]. Same end-character rules: unclosed\Qconsumes to]; metacharacters lose their meaning.- Note:
\Eoutside a\Q…\Eblock is an “Illegal/unsupported escape sequence” error.
§1.2 Predefined character classes
| Class | Default (ASCII) meaning | With UNICODE_CHARACTER_CLASS (U) flag |
|---|---|---|
\d | [0-9] | Any character in Unicode general category Nd (Decimal_Number) |
\D | [^0-9] | Complement of \d |
\w | [a-zA-Z0-9_] | Letters, digits, marks, joining _ (a curated Unicode set per the Javadoc) |
\W | Complement of \w | Complement of \w |
\s | [ \t\n\x0B\f\r] | [\p{IsWhite_Space}] |
\S | Complement of \s | Complement of \s |
\h | [ \t\xA0 - ] | Same (horizontal whitespace; Unicode by definition) |
\H | Complement of \h | Same |
\v | [\n\x0B\f\r\x85
] | Same (vertical whitespace; Unicode by definition) |
\V | Complement of \v | Same |
\R is not a predefined character class — it is a line-break matcher that matches the two-character sequence \r\n or any single character in [\n\x0B\f\r\x85
]. Crucially:
- Inside a quantifier (e.g.,
\R{2},\R+),\Ris atomic — the engine never backtracks into it to try the single-character branch after the two-character branch fits. So\R{2}on"\r\n"does not match: iter 1 consumes\r\n, iter 2 has nothing left, and no backtrack is attempted. [Q-2, Q-3] - In a sequence outside a quantifier (e.g.,
\R\n),\Rdoes backtrack into its branches.
\X matches one extended grapheme cluster (per UAX #29), including ZWJ-joined sequences like “👨👩👦”. \X is unbounded in length, so it is rejected in lookbehind bodies (see §1.7).
§1.3 Character classes ([...])
A character class matches a single character drawn from a set built by union, range, intersection, and negation.
Basic forms.
| Form | Meaning |
|---|---|
[abc] | a, b, or c |
[^abc] | Any character except a, b, c |
[a-z] | Any character in the inclusive range U+0061..U+007A |
[a-zA-Z0-9_] | Union of three ranges and one literal |
[[a-z][A-Z]] | Union of two nested classes (= [a-zA-Z]) |
[a-z&&[def]] | Intersection: {a..z} ∩ {d,e,f} = {d,e,f} |
[a-z&&[^aeiou]] | Set difference (intersection with complement) |
Special characters inside [...]. Unlike outside, most metacharacters lose their special meaning inside a character class:
\is still an escape character.]ends the class. To match a literal], place it first (after[or[^):[]a]matches]ora. Or escape it:[\]a].-is the range operator between two characters; literal-is matched if placed first, last, or escaped.^immediately after[negates the class. Elsewhere it is literal.&&is the intersection operator (see below). A single&is literal.[opens a nested class.\Q…\Eis supported; see §1.1.
Predefined classes inside [...]. \d, \w, \s, \D, \W, \S, \h, \H, \v, \V, \p{…}, and \P{…} may all appear inside [...] and contribute their set membership.
Ranges. The range operator - is between two single characters (literal or escaped). The start must be ≤ the end ([z-a] is rejected). Predefined classes (\d, etc.) cannot be range endpoints; using one — e.g., [\d-x] — is not a range, it is the literal three-element union \d ∪ {-} ∪ {x} (the - is treated literally when adjacent to a non-single-character item).
Negation. ^ immediately inside [ negates the entire class. [^a-z] is “anything not in a..z.” Negation applies to the union/intersection result, not to individual items.
Intersection &&. Computes the set intersection of the left and right operands:
[abc&&[def]]is empty (no overlap).[a-z&&[^aeiou]]is “lowercase letters minus vowels.”[a-z&&[A-Z]&&xyz]is not a clean three-way intersection. See [Q-4] for the quirk: when the middle operand contains a nested class followed by a literal, the trailing&&is silently absorbed into the recursive sub-class scope. Workaround: use explicit nesting[[a-z&&[A-Z]]&&xyz].
Nested character classes. [[a-z][A-Z]] is the union of two nested classes (= [a-zA-Z]). Nesting is supported to arbitrary depth.
Case-folded ranges. When CASE_INSENSITIVE is active, range membership checks the input character and its case variant against the unmodified range — not the folded range. So [1-c] with /i matches 'g' because 'G' (U+0047) is inside the range U+0031..U+0063. [Q-6]
§1.4 Quantifiers
A quantifier follows an atom and specifies how many times that atom must match.
| Quantifier | Min | Max | Notes |
|---|---|---|---|
X? | 0 | 1 | Optional |
X* | 0 | ∞ | Zero or more |
X+ | 1 | ∞ | One or more |
X{n} | n | n | Exactly n |
X{n,} | n | ∞ | At least n |
X{n,m} | n | m | Between n and m (inclusive) |
Each form has three modes, distinguished by a suffix:
| Suffix | Mode | Behavior |
|---|---|---|
| (none) | Greedy | Take as many as possible, backtrack to free up characters for the rest of the pattern |
? | Reluctant (lazy) | Take as few as possible, expand only if rest fails |
+ | Possessive | Take as many as possible, never backtrack (no give-back) |
Examples: X*? (reluctant *), X*+ (possessive *), X{2,5}? (reluctant range), X{2,5}+ (possessive range).
Possessive semantics. A possessive quantifier matches like a greedy one but commits to its choice. If the rest of the pattern fails after a possessive match, the engine does not retry with fewer matches; the whole pattern fails (or backtracks to a point before the possessive).
Quantified atom compilation. OpenJDK compiles a quantified atom with one of two strategies, chosen at parse time:
GroupCurly/Curly(atomic body) — when the body is deterministic (single top-level branch, no nested variable-count quantifier). The atom is matched in isolation; the engine does not backtrack into the atom’s internal choices.Loop/LazyLoop(chain body) — when the body has alternation, a Ques quantifier (X?/X??), or another non-deterministic shape. The continuation is threaded through the body so the engine can backtrack into the atom.
Consequence: \R{2} on "\r\n" does not match. \R is a single atom whose two-character branch always wins; with the body deterministic, no backtrack into \R is attempted. (?:a|aa){2} on "aaa" does match — alternation forces the chain compilation. [Q-2, Q-3]
The atomic split is observable through any wrapper: \R{2}, (?:\R){2}, (\R){2}, (?>\R){2}, (?i:\R){2} all behave identically (all atomic). Only an internal alternation or a nested variable-count quantifier flips to the chain compilation.
Zero-width quantified body. If the body matches zero characters (e.g., ()* or (?=a)*), the engine treats subsequent zero-width iterations as having already reached the maximum count and proceeds to the rest of the pattern. This avoids infinite loops on (\b)*-style patterns.
A capturing group with a * or {0,N} quantifier whose body matched zero times (cmin == 0 and the body never executed) leaves the group null, not the empty string. Compare: (), ()+, (\Q\E), (a*)* all set group 1 to "" (the body did run, possibly matching empty); but (\Q\E)*, ()* leave group 1 null.
§1.5 Groups
A group is a sub-expression delimited by ( and ). OpenJDK supports six kinds:
| Syntax | Kind | Captures? | Inherits flags? |
|---|---|---|---|
(X) | Capturing | Yes (numbered, 1-based, left-paren order) | Yes |
(?<name>X) | Named capturing | Yes (numbered AND named) | Yes |
(?:X) | Non-capturing | No | Yes |
(?>X) | Atomic (non-capturing) | No | Yes |
(?flags:X) or (?flags-flags:X) | Flag group (non-capturing) | No | Modifies flags locally |
(?flags) or (?flags-flags) | Inline flag setter (no body, no parens around content) | No | Modifies flags in enclosing scope |
Group numbering. Capturing groups are numbered 1, 2, 3, … in the order their opening ( appears in the pattern. Group 0 is always the whole match. Non-capturing kinds (?:, ?>, ?=, ?!, ?<=, ?<!, ?flags:) do not receive a number and do not affect downstream numbering.
Named groups. A group name is [A-Za-z][A-Za-z0-9]* (must start with a letter; subsequent characters letter or digit; no Unicode names). Named groups are also numbered — (?<a>x)(?<b>y) gives group 1 = a, group 2 = b. Duplicate names within one pattern are a compile error.
Atomic groups (?>X). Once X succeeds, the engine commits and will not backtrack into X even if the rest of the pattern fails. Atomic groups are non-capturing and do not propagate inline flags out of the group (see §2).
Flag groups (?flags:X). The flags listed in flags are added to the active flag set for the duration of X. A - after the letters subtracts: (?-i:X) turns off i for X. Compound: (?im-s:X) enables i and m, disables s for X. After the closing ), the original flags are restored.
Inline flag setter (?flags) / (?flags-flags). No colon, no body. The setter mutates the parser’s flag state from that point onward, scoped to the enclosing group (or the whole pattern if at the top level). Top-level inline flags propagate across alternation branches; wrapping in any group (including (?:...), (...), (?>...), (?=...)) scopes the change to that group. [Q-5]
(?s)|. on "\n" → matches: true (top-level (?s) leaks into alt 2)
(?s)xx|. on "\n" → matches: true (even from a failing branch)
(?:(?s))|. on "\n" → matches: false (wrapped → scoped)
((?s))|. on "\n" → matches: false (wrapped → scoped)
(?>(?s))|. on "\n" → matches: false (wrapped → scoped)This is a compile-time behavior, not runtime: the parser mutates its own flag-state when it encounters (?…) and the change persists to subsequent branches parsed afterwards — regardless of whether those branches end up matching at runtime.
§1.6 Anchors and boundaries
Anchors match a position, not a character.
| Anchor | Position |
|---|---|
^ | Start of input (or start of line, if MULTILINE) |
$ | End of input or before final line terminator (or end of line, if MULTILINE) |
\A | Start of input (regardless of MULTILINE) |
\z | End of input (regardless of MULTILINE) |
\Z | End of input or before final line terminator (regardless of MULTILINE) |
\b | Word boundary (transition between \w and \W, or input edge) |
\B | Not a word boundary |
\G | End of the previous match (or start of input for the first find() call) |
Line terminator. By default, “line terminator” means any of \n, \r, \r\n, U+0085 NEL, U+2028 LS, or U+2029 PS. With UNIX_LINES ((?d) or Pattern.UNIX_LINES), only \n is a line terminator.
Multiline ^. With MULTILINE, ^ matches at the start of input and immediately after every line terminator — except at the very end of input, even if the end of input is also the end of a line. So:
^ on "" (MULTILINE) → no match
^ on "\n" (MULTILINE) → matches at pos 0 only (NOT at pos 1)
^ on "\r\né" (MULTILINE) → matches at positions 0, 1, 2 onlyThis mirrors a Perl quirk. [Q-1]
$ in non-multiline mode. $ matches at end of input, or just before a single line terminator at end of input (so the pattern ^abc$ matches "abc\n"). With UNIX_LINES, “single line terminator” means \n only.
\Z. Same as $ but unaffected by MULTILINE.
\z. Strict end of input; does not match before a trailing newline.
Word boundary \b. A boundary exists between two adjacent character positions if exactly one of them has \w and the other has \W (or is past an input edge). With UNICODE_CHARACTER_CLASS active, “word character” expands to the Unicode word set; otherwise it is ASCII [a-zA-Z0-9_].
\G. For find(), anchors at the position where the previous match ended (or start of input for the first call). For matches() and lookingAt(), anchors at the search start position. Useful for tokenizers: \G\w+ finds successive word tokens with no gaps.
Region boundaries. When Matcher.region(start, end) is set:
- By default (anchoring bounds),
^,\A,$,\z,\Ztreat the region as if it were the whole input.\bsimilarly treats positions at region edges as input edges. - With
useAnchoringBounds(false), anchors and\bsee the real input boundaries. - With
useTransparentBounds(true), lookarounds can look across region edges into the rest of the actual input. By default (useTransparentBounds(false)), lookarounds are clipped at the region. \Zand$(non-multiline) consult the character before the current position. They do not honor region bounds for this character lookup — they always look at the underlying input character. So\Zat position 1 of region[1, 2)of input"\r\n\r"sees the\rat position 0 (outside the region) and correctly identifies that position 1 is inside\r\n(so\Zdoes not match here).
§1.7 Lookarounds
A lookaround matches a position by attempting an inner pattern at that position without consuming characters.
| Syntax | Direction | Polarity |
|---|---|---|
(?=X) | Lookahead | Positive (require X) |
(?!X) | Lookahead | Negative (require not X) |
(?<=X) | Lookbehind | Positive (require X immediately before) |
(?<!X) | Lookbehind | Negative (require not X immediately before) |
Lookahead. Attempts to match X starting at the current position. Position is not consumed. Captures inside X are written to the matcher’s group state — and they persist even when the lookahead is negative and the inner match succeeded (which makes the overall lookaround fail). See §4 on capture-state leaks. [Q-8]
Lookbehind. Attempts to match X ending exactly at the current position. The inner pattern is run “backwards” — OpenJDK enumerates all positions j in [i - rmax, i - rmin] and attempts to match X against the slice input[j..i].
Bounded-length requirement. Lookbehind requires that the inner pattern have a statically known maximum length. OpenJDK’s compile-time check uses TreeInfo.maxLength over the inner AST. Patterns that fail this check are rejected with "Look-behind group does not have an obvious maximum length". Examples:
(?<=a) OK (max len 1)
(?<=ab) OK (max len 2)
(?<=a{2,4}) OK (max len 4)
(?<=a+) OK (single-char unbounded is fine — body max len is "really" 1)
(?<=(?:ab)+) REJECTED (multi-char unbounded)
(?<=ab*) REJECTED (mixed; total is unbounded)
(?<=\R+) REJECTED (\R is multi-char)
(?<=\X) REJECTED (\X is unbounded)
(?<=\1) REJECTED (backref has no compile-time length)The single-char-unbounded exception applies to character-class repeats: a+, \d*, [ab]+ all give a body max length the engine clamps to a single-character iteration count. Multi-character repeats ((?:ab)+, \R+) don’t get this exception.
i32 wrapping in lookbehind body sizing. OpenJDK’s TreeInfo.maxLength uses int arithmetic with overflow (no saturation). A *+ possessive contributes MAX_REPS (= 0x7FFFFFFF); enclosing context can push the body’s max length past Integer.MAX_VALUE and into a negative value. The engine then computes the iteration window [i - rmax, i - rmin] with that negative rmax, which makes i - rmax very large — effectively skipping body iteration when the overflow is enough. This is a documented quirk we reproduce; certain (?<!...) constructs with large nested counts will accept where a naive non-overflow implementation would reject. (See pattern_java_max in the source.)
Negative lookbehind with Branch alternation. When the body has an alternation (empty | unbounded), Java’s Branch.study takes Math.max(0, neg_overflowed) across atoms. The empty alt contributes 0, which dominates the negative wrap. Net effect: rmax = 0, and the engine iterates j = i (zero-width), letting the empty alt match. So (?<!|...) with an unbounded alt 2 still fails the negative lookbehind because the empty alt matches.
Lookaround capture state. Captures set inside a lookaround’s inner pattern persist into the outer matcher state even if the lookaround is negative and the inversion makes the outer match fail. See §4. [Q-8]
§1.8 Backreferences
A backreference matches the same characters as a previously-captured group.
| Syntax | Meaning |
|---|---|
\1 \2 … \9 | Reference to group N (1-9) |
\N for N ≥ 10 | Reference to group N if N ≤ total group count; otherwise greedy fallback |
\k<name> | Reference to named group |
Multi-digit number semantics. OpenJDK consumes digits greedily but caps at the actual group count. With 12 capture groups, \12 is a reference to group 12. With 5 capture groups, \12 parses as \1 followed by literal 2.
Case-sensitivity. When CASE_INSENSITIVE is active, the backreference compares case-insensitively character-by-character. With UNICODE_CASE active, the comparison uses Unicode case folding (per-character toLowerCase/toUpperCase); otherwise it uses ASCII case folding.
Backreferences with capture groups not yet executed. If \N references a group that has not yet successfully captured (e.g., the group is in an unmatched alternation branch, or the backreference is inside the group itself), the captured value is null. A backreference to a null capture fails to match — it is not equivalent to an empty match.
Backreferences to zero-width captures. A backreference to a group that captured the empty string (e.g., (?:), (?=.)) matches zero characters and succeeds.
Empty-capture backreference in a quantifier. Curly detects i == matcher.last (zero-width iteration) and short-circuits to the next iteration’s continuation. This is how (.*)\1+ does not loop forever when the first .* captures empty.
Backreferences inside lookbehind bodies are rejected (lookbehind requires bounded length; backref is runtime-sized).
§1.9 Unicode properties
\p{Name} matches a character whose Unicode property is Name; \P{Name} is the complement.
Property categories. OpenJDK accepts:
-
General Categories by full name (
\p{Lu},\p{Letter},\p{Uppercase_Letter}), short name, or both:L/Letter,Lu/Uppercase_Letter/Upper,Ll/Lowercase_Letter/Lower,Lt,Lm,Lo,Lc(cased letter)M/Mark,Mn,Mc,MeN/Number,Nd/Digit,Nl,NoP/Punctuation/Punct,Pc,Pd,Ps,Pe,Pi,Pf,PoS/Symbol,Sm,Sc,Sk,SoZ/Separator,Zs,Zl,ZpC/Control/Other,Cc,Cf/Format,Co,Cn
-
POSIX classes (case-insensitive name):
Alpha,Lower,Upper,Digit,Alnum,Punct,Graph,Print,Blank,Cntrl,XDigit,Space,ASCII- These have two modes: ASCII-only (default) and Unicode (when
UNICODE_CHARACTER_CLASSis active). The Unicode variants of POSIX classes redefine each to its Unicode equivalent (e.g.,\p{Lower}becomes any character withLowercase=True).
-
Special Java properties (exact case,
javaprefix):javaLowerCase,javaUpperCase,javaTitleCase,javaDigit,javaLetter,javaLetterOrDigit,javaAlphabetic,javaWhitespace,javaSpaceChar,javaMirrored,javaDefined,javaIdentifierIgnorable,javaISOControl,javaUnicodeIdentifierStart,javaUnicodeIdentifierPart- Each corresponds to a
java.lang.Character.isXxx()predicate.
-
Scripts with
Isprefix:\p{IsLatin},\p{IsGreek},\p{IsCyrillic}, etc. Both ISO 15924 full names (IsLatin) and short codes (IsLatn) are accepted. -
Blocks with
Inprefix:\p{InBasicLatin},\p{InGreek},\p{InCJKUnifiedIdeographs}, etc. The block names follow Unicode’s UAX #44 block names with spaces and underscores stripped. -
\p{L1}/\p{Latin1}— characters U+0000..U+00FF.
Property syntax variants.
\pX → \p{X} for X a single letter (\pL, \pN, ...)
\p{X} → property X
\p{IsLatin} → script Latin
\p{InGreek} → block Greek
\p{name=value} → property with named value (rarely used; OpenJDK accepts only specific forms)
\p{^X} → equivalent to \P{X}Property names are matched case-sensitively for the “Is” / “In” / “java” prefixes; categories like Lu are case-sensitive (short form is exact case); long names like Uppercase_Letter are case-sensitive.
\p{Mirrored} math symbols. Java’s Character.isMirrored() is the Bidi_Mirrored Unicode property. For math symbols (Sm), OpenJDK uses a curated subset: ∈ (U+2208), ≤ (U+2264), ⊏ (U+228F), etc. are mirrored; ∀ (U+2200) is not. ASCII < and > are special-cased as mirrored.
§2. Flags and inline flag scoping
Flags can be set at compile time (Pattern.compile(pat, flags)), inline ((?flags)), or scoped to a group ((?flags:body)).
| Flag | Letter | Pattern. constant | Effect |
|---|---|---|---|
| CASE_INSENSITIVE | i | CASE_INSENSITIVE | Match letters case-insensitively (ASCII only unless u is also on) |
| MULTILINE | m | MULTILINE | ^ and $ match at every line break, not just input edges |
| DOTALL | s | DOTALL | . matches line terminators too |
| COMMENTS | x | COMMENTS | Strip unescaped whitespace and # … EOL comments from the pattern |
| UNICODE_CASE | u | UNICODE_CASE | Case-insensitivity uses Unicode case folding (not just ASCII) |
| UNICODE_CHARACTER_CLASS | U | UNICODE_CHARACTER_CLASS | \d, \w, \s, POSIX classes, \b use full Unicode sets |
| UNIX_LINES | d | UNIX_LINES | Only \n is a line terminator (., ^, $, \Z) |
| LITERAL | — | LITERAL | Treat the whole pattern as a literal string (compile-time only) |
| CANON_EQ | — | CANON_EQ | (Not implemented; not in scope of this spec) |
Letter ⇔ constant correspondence. Inline (?i) is CASE_INSENSITIVE, (?m) is MULTILINE, and so on. There is no inline letter for LITERAL or CANON_EQ (those are constructor-time only).
Inline flag scoping rules.
- A bare
(?flags)at the top level persists to the end of the pattern and propagates across|alternation branches that come after it. Branches parsed before it do not see it. - A bare
(?flags)inside any group ((...),(?:...),(?>...), etc.) is scoped to that group — the flag change reverts at the closing). - A scoped flag group
(?flags:X)applies the flags only toX. (?flags-clear)addsflagsand removesclear. Either side may be empty:(?-x)only clears.
Case-insensitivity (i, u, U interaction).
ialone: ASCII case folding (A-Z↔a-z).iu(or(?iu)): Unicode case folding.Ämatchesä, etc.U(UNICODE_CHARACTER_CLASS): Expands the sets\d/\w/\s/POSIX classes /\bto Unicode. Does NOT by itself enable case-folding.iU: Unicode predefined classes but ASCII case folding (rarely useful —Uimplies wanting Unicode behavior, soiuUis the common combination).
§3. Matching operations
java.util.regex.Matcher exposes several match operations with subtly different semantics.
§3.1 matches()
Attempts to match the entire input (subject to the region, if set). Equivalent to running the pattern with implicit \A...\z anchoring (over the region).
- Returns
trueiff the whole input from the region start to the region end is consumed by a single match attempt. - Captures from a successful match are stored.
- A failed
matches()may leave partial captures from internal sub-paths that succeeded. (See §4.)
§3.2 find()
Scans the input for the next match starting at the current search position (initially region start, or end of previous match for subsequent calls). Returns true iff a match is found.
Position iteration. OpenJDK’s Start.match iterates the search position from the current cursor through to (and including) the end of input. At each position, it attempts the pattern; on success, the match ends; on failure, the position advances by one and the attempt repeats. The groups[] capture array is not reset between attempts. [Q-8]
Empty match advance. If a find() returns a zero-width match at position N, the next find() starts at N + 1 to prevent infinite loops.
§3.3 lookingAt()
Attempts to match starting at the current search position (initially the region start). The match need not consume the entire input — it just needs to succeed starting at position 0.
- Equivalent to
find()restricted to starting at the search start position only. - Useful for “tokenize while pattern matches at the cursor” workflows together with
\G.
§3.4 Region
Matcher.region(start, end) constrains all subsequent matches to the half-open range [start, end). Position 0-based, UTF-16 code-unit indices.
region(start, end)resets the matcher state (search position, captures, etc.).regionStart()/regionEnd()return the current region.useAnchoringBounds(false)makes^,\A,$,\z,\Z,\bignore region boundaries and see the actual input edges.useTransparentBounds(true)makes lookarounds able to peek outside the region.- Some character-context anchors (
\Zlooking back at the previous char for\r\n) deliberately bypass the region. This is OpenJDK’s documented behavior and matters when constructing patterns that consult the boundary character.
§3.5 hitEnd() and requireEnd()
After a failed or successful match:
hitEnd()returnstrueif the matcher consumed input up to (or past) the region end during the attempt — i.e., the failure or success “touched” the end. Useful for incremental matching: a failed match withhitEnd()true means “maybe just need more input.”requireEnd()returnstrueif a successful match could have been longer given more input — i.e., the match’s end position is at the region end and adding characters could extend it. Useful for streaming matchers.
§4. Capture-group state model
This is the part of Pattern / Matcher semantics least well-described by the Javadoc.
State storage. The matcher keeps a single integer array groups[] of size 2 * (groupCount + 1), holding (start, end) pairs for groups 0..N. A capture is considered “set” iff its start is non-negative.
State persistence across constructs.
Branch.match(alternation) — Does not save/restore between branches. If branch 1 captures and then later fails (because the rest of the pattern fails after branch 1’s commit), captures from branch 1 persist into branch 2’s attempt.Start.match(find-position loop) — Does not resetgroups[]between starting-position attempts. Captures from a failed position-N attempt leak into the successful match at a later position. [Q-8]Lookahead((?=X)and(?!X)) — Does not save/restore. Inner captures persist whether the lookahead succeeds or fails (and whether the polarity inverts it).NotBehind((?<!X)) — Does not save/restore. If the inner X matches (which then flips the outer to failure), X’s captures persist.Behind((?<=X)) — Does not save/restore around the inner.GroupTail(per-capturing-group end marker) — Does save/restore the group’s own slot when the rest of the pattern fails after the group successfully completed. This is the only construct that restores capture state.Curly/GroupCurly(deterministic quantifier) — Within one iteration, the inner GroupTails do not see the outer continuation; they are commits. The outer GroupCurly explicitly re-stampsgroups[idx] = (i - k, i)for capturing-group atoms afternext.matchsucceeds. So(X){3}captures the last iteration ofX.Loop/LazyLoop(non-deterministic quantifier) — Threads the continuation through the body. Inner GroupTails see the full continuation and restore on downstream failure. So(a|b){3}captures the last iteration’s value, which is the value at the latest successful chain unwind.
Consequences (with examples).
Pattern: (?=(\w))*\s
Input: "a "
find()[0]: start=1, end=2, text=" ", g1="a" ← captures from failed pos-0 leakPattern: (?<!(a|bb))c?
Input: "ac"
find()[1]: start=2, end=2, text="", g1="a" ← inner cap from failed pos-1 negative LBPattern: (?:([^\w])+){2}
Input: "\t\t\r"
find()[0]: text="\t\t\r", g1="\t" ← outer GroupCurly's backoff re-stamps g1The third example illustrates the “GroupCurly backoff re-stamp”: the inner ([^\w])+ matches once consuming \t\t\r (all 3 chars). The outer Loop then needs a second iteration but no chars remain. The inner GroupCurly’s backoff loop tries next.match at successively smaller match counts; when it finds one that fits (count = 1, \t at position 0), it explicitly re-stamps groups[1] = (0, 1) = "\t" based on its current backoff slice. The recursive 2nd iteration sets groups[1] = (1, 2) = "\t", but the outer GroupCurly’s re-stamp at iteration 1 wins on chain unwind. Net effect: g1 = "\t" (slice (0,1)).
Practical rule. Capture state should be read only after a successful match. Reading after a failed match — or between branches — yields partial, attempt-dependent values. The persistence is observable and forms part of the spec; it is not undefined behavior, just often surprising.
§5. split()
Pattern.split(input) and Pattern.split(input, limit) split the input at every non-overlapping occurrence of the pattern.
Algorithm. Find matches left-to-right with find(). Between matches, emit the text between the previous match’s end (or start of input) and the current match’s start. After the last match, emit the trailing text.
Zero-width match suppression at position 0. If the very first match is zero-width and at position 0, the would-be leading empty string is suppressed. This means:
\Q\E split "abc" → ["a", "b", "c"]
\Q\E split "\t" → ["\t"]
(?=b) split "abc" → ["a", "bc"]
a split "abc" → ["", "bc"] ← non-zero-width → leading empty keptThe Javadoc covers this rule; it is easy to miss the first time. [Q-7]
Limit parameter.
limit > 0: At mostlimit - 1splits performed. The last element contains all remaining input (including any later matches as literal text).limit == 0: Default. Splits as many times as possible, but trailing empty strings are removed.limit < 0: Splits as many times as possible; trailing empty strings are retained.
"a,b,,".split(",") → ["a", "b"] (limit=0, trailing empties removed)
"a,b,,".split(",", -1) → ["a", "b", "", ""] (trailing empties kept)
"a,b,,".split(",", 2) → ["a", "b,,"] (limit=2, at most 1 split)§6. Replacement DSL (appendReplacement / replaceAll)
The replacement string in Matcher.appendReplacement(sb, replacement) and Matcher.replaceAll(replacement) uses its own mini-language.
Tokens.
| Token | Meaning |
|---|---|
\$ | Literal $ |
\\ | Literal \ |
\X (any other X) | Literal X (the \ is consumed) |
$0 | The whole match |
$N (N = 1..9 or more) | Group N’s captured text |
${name} | Named group’s captured text |
Multi-digit $N rule. The reader is greedy: it consumes digits until the resulting number would exceed the actual group count. So with 5 groups, $12 is $1 followed by literal 2; with 12 groups, $12 is group 12.
Null captures. If group N matched but its value is null (group not executed in the chosen branch), the replacement inserts the empty string.
Literal $ and \ in the replacement. Use \$ for $ and \\ for \. Bare $ followed by a non-digit / non-{ is also treated as literal $ (Java tolerates this — though for safety, escape it).
replace_all("a", r"\$") → "$"
replace_all("a", "$$") → "$$" (bare $ before non-digit → literal $)Matcher.quoteReplacement(s) returns s with every $ and \ doubled (escaped), useful for inserting arbitrary text without DSL interpretation.
§7. Compile-time errors
OpenJDK’s Pattern.compile raises PatternSyntaxException for the following invalid constructs. The message is <description> near index <N>\n<pattern>\n<padding>^ where ^ points at the offending position.
| Error | Trigger |
|---|---|
Unmatched closing brace ) | ) without preceding ( |
Unmatched closing brace } | } outside {n,m} quantifier or ${name} |
Unmatched closing bracket ] | ] outside […] |
| Unclosed group | ( without matching ) |
| Unclosed character class | [ without matching ] |
Unknown character property {X} | \p{X} with unrecognized X |
| Look-behind group does not have an obvious maximum length | Lookbehind body has unbounded length |
| Illegal repetition | Quantifier {n,m} with m < n, or {} with no digits |
| Dangling meta character | *, +, ? not preceded by an atom |
| Bad backref | \N for N > group count + escape ambiguity |
| Illegal/unsupported escape sequence | \E outside \Q…\E; \N{…} |
| Empty intersection operand | […&&] or [&&…] |
| Illegal character range | [c-c-] where the second - would create an invalid range |
| Invalid hex escape | \xZZ, \u123, unclosed \x{…} |
| Duplicate name | Two named groups with the same name |
| Unknown named group | \k<name> or ${name} referring to a name not defined in the pattern |
The exact message text varies between Java versions; the condition is stable.
§8. Quirks index
For each documented divergence from a literal Javadoc reading, see QUIRKS.md:
^at end of input — multiline^never matches at the very end of input, even after a trailing line terminator.- Deterministic atom atomicity —
\R{2},(?:\R){2},(?i:\R){2}are atomic. \Rbacktracks in sequence but not in a quantifier — direct consequence of #2.- Chained
[A && B && C]— drops trailing clause when the middle operand contains a nested class followed by a literal. - Inline
(?s)propagates across alternation, but only at the top level. [1-c]/imatchesg— case-insensitive range membership uses input char vs unmodified range.split()suppresses leading empty for a zero-width match at position 0.- Capture state leaks across find positions and from failed lookarounds.
For intentional deviations of this Rust port from OpenJDK, see DIFFERENCES.md (one item: UTF-16 vs UTF-8 position offsets).
§Appendix A: Cross-engine porting tips
- From PCRE / Perl. OpenJDK is close but not identical. Notable differences: no
\K(keep-out), no recursion ((?R)), no conditional ((?(cond)yes|no)), no possessive at top level (atomic(?>...)exists), no(*VERB)control verbs. Property syntax\p{…}is supported but not all PCRE-specific properties. - From
regex(Rust crate). Java has lookahead/lookbehind, backreferences, atomic groups, possessive quantifiers — all of whichregexdeliberately excludes. The Rust port (java_regex) supports the full surface. - From Oniguruma / Onigmo. Most of OpenJDK’s syntax overlaps.
\g<name>(subroutine call) does not exist in Java. Named-group syntax(?<name>...)is identical.
§Appendix B: Pattern.compile vs Matcher
Pattern is the compiled, immutable, thread-safe representation of a regex. Matcher is a per-input scratchpad: it holds the input reference, the current search position, the region bounds, and the groups[] capture array. Creating one Pattern and reusing it across many Matcher instances is the standard idiom and is thread-safe; reusing one Matcher across threads is not.