# HTML parsing conformance
How closely domq's trees match a browser's, measured with the HTML
tree-construction tests (the html5lib tests, maintained in web-platform-tests)
at commit [`4aa56d3`](https://github.com/web-platform-tests/wpt/tree/4aa56d3beafc5655fa55491169eb14845b44eab3/html/syntax/parsing/resources).
**1959 tests run**, 14 skipped (they need scripting enabled), **0 panics**.
| Exact tree: the html5lib dump, line for line | 83 | 4.2% |
| Same structure, in domq's model of HTML | 1210 | 61.8% |
domq is a forgiving parser, not a spec-compliant one (see the
[README](../README.md#what-it-handles)), so the exact rate is low by design:
most expected trees start with `<html>`, `<head>` and `<body>` elements that a
browser inserts and domq does not. **Same structure** compares the trees after
removing what domq deliberately does differently:
- implied `<html>`, `<head>`, `<body>` and `<tbody>` are unwrapped (their
children take their place) in both trees, and so is a template's `content`;
- namespaces and the case of SVG and MathML names are ignored
(`<svg foreignObject>` and `<foreignobject>` are equal);
- whitespace-only text nodes are dropped, and adjacent text is merged;
- doctypes compare by name only.
Everything else counts: which elements exist, how they nest, their attributes,
text and comments. A test that fails here is a real difference from a browser.
## By area
| General tree building | 647 | 1.5% | 66.0% |
| Comments, doctypes, quirks | 87 | 0.0% | 81.6% |
| Character references | 101 | 0.0% | 100.0% |
| Raw text, script data, unsafe text | 380 | 0.0% | 55.5% |
| Mis-nested formatting (adoption agency) | 86 | 0.0% | 26.7% |
| Tables and foster parenting | 63 | 0.0% | 41.3% |
| Foreign content (SVG, MathML) | 202 | 19.3% | 71.3% |
| Templates | 127 | 1.6% | 73.2% |
| Fragments and context | 142 | 22.5% | 52.1% |
| Processing instructions (new in the spec) | 124 | 0.0% | 32.3% |
### Where the gaps are by design
- **Mis-nested formatting**: browsers run the adoption agency algorithm on
`<b><p>x</b>y</p>`, cloning formatting elements across blocks, and reopen
formatting elements a block closed; domq closes the formatting element and
moves on. A bounded version is the next item on the
[roadmap](../ROADMAP.md).
- **Tables**: browsers move stray content out of tables (foster parenting)
and insert `<tbody>`; domq keeps content where it appears. Optional
`<tbody>` insertion is planned for 0.7.
- **Foreign content**: domq has no namespaces, and lower-cases every tag name.
- **Templates**: a template's children stay its children (browsers move them
into a separate document fragment, `content`); the structure score unwraps
`content` for this reason.
- **Fragments**: domq parses a fragment like a document, without the context
element a test names (`<td>` parsed inside a `<tr>`). Context-aware
fragment parsing is planned for 0.7.
- **Scripting**: domq never runs scripts, so tests that need scripting enabled
are skipped; `<noscript>` content is parsed as markup.
- **Quirks mode**: domq always builds the tree as in standards mode (a
`<table>` closes an open `<p>`, as with `<!DOCTYPE html>`).
- **Processing instructions**: a recent change to the HTML standard keeps
`<?target data>` as a processing-instruction node; domq, like the browsers
that predate it, turns it into a comment.
## By file
| `adoption01.dat` | Mis-nested formatting (adoption agency) | 18 | 0.0% | 11.1% |
| `adoption02.dat` | Mis-nested formatting (adoption agency) | 4 | 0.0% | 0.0% |
| `blocks.dat` | General tree building | 48 | 0.0% | 100.0% |
| `comments01.dat` | Comments, doctypes, quirks | 16 | 0.0% | 100.0% |
| `doctype01.dat` | Comments, doctypes, quirks | 37 | 0.0% | 97.3% |
| `domjs-unsafe.dat` | Raw text, script data, unsafe text | 49 | 0.0% | 40.8% |
| `entities01.dat` | Character references | 75 | 0.0% | 100.0% |
| `entities02.dat` | Character references | 26 | 0.0% | 100.0% |
| `foreign-fragment.dat` | Foreign content (SVG, MathML) | 66 | 59.1% | 86.4% |
| `html5test-com.dat` | Comments, doctypes, quirks | 30 | 0.0% | 60.0% |
| `inbody01.dat` | General tree building | 4 | 0.0% | 100.0% |
| `isindex.dat` | General tree building | 4 | 0.0% | 100.0% |
| `main-element.dat` | General tree building | 3 | 0.0% | 66.7% |
| `math.dat` | Foreign content (SVG, MathML) | 8 | 0.0% | 62.5% |
| `menuitem-element.dat` | General tree building | 20 | 0.0% | 90.0% |
| `namespace-sensitivity.dat` | Foreign content (SVG, MathML) | 1 | 0.0% | 0.0% |
| `noscript01.dat` | Raw text, script data, unsafe text | 18 | 0.0% | 55.6% |
| `pending-spec-changes-plain-text-unsafe.dat` | General tree building | 1 | 0.0% | 0.0% |
| `pending-spec-changes.dat` | General tree building | 3 | 0.0% | 66.7% |
| `plain-text-unsafe.dat` | Raw text, script data, unsafe text | 53 | 0.0% | 1.9% |
| `processing-instructions.dat` | Processing instructions (new in the spec) | 124 | 0.0% | 32.3% |
| `quirks01.dat` | Comments, doctypes, quirks | 4 | 0.0% | 25.0% |
| `ruby.dat` | General tree building | 21 | 0.0% | 100.0% |
| `scriptdata01.dat` | Raw text, script data, unsafe text | 26 | 0.0% | 61.5% |
| `scripted_adoption01.dat` | Mis-nested formatting (adoption agency) | 0 | — | — |
| `scripted_ark.dat` | Mis-nested formatting (adoption agency) | 0 | — | — |
| `scripted_foster01.dat` | Tables and foster parenting | 0 | — | — |
| `scripted_webkit01.dat` | General tree building | 0 | — | — |
| `search-element.dat` | General tree building | 8 | 0.0% | 50.0% |
| `svg.dat` | Foreign content (SVG, MathML) | 8 | 0.0% | 0.0% |
| `tables01.dat` | Tables and foster parenting | 19 | 0.0% | 21.1% |
| `template.dat` | Templates | 127 | 1.6% | 73.2% |
| `tests1.dat` | General tree building | 112 | 4.5% | 60.7% |
| `tests10.dat` | Foreign content (SVG, MathML) | 54 | 0.0% | 61.1% |
| `tests11.dat` | Foreign content (SVG, MathML) | 13 | 0.0% | 100.0% |
| `tests12.dat` | Foreign content (SVG, MathML) | 2 | 0.0% | 100.0% |
| `tests14.dat` | General tree building | 7 | 0.0% | 100.0% |
| `tests15.dat` | Raw text, script data, unsafe text | 14 | 0.0% | 35.7% |
| `tests16.dat` | Raw text, script data, unsafe text | 191 | 0.0% | 73.8% |
| `tests17.dat` | Raw text, script data, unsafe text | 13 | 0.0% | 38.5% |
| `tests18.dat` | General tree building | 36 | 0.0% | 50.0% |
| `tests19.dat` | General tree building | 107 | 0.0% | 39.3% |
| `tests2.dat` | General tree building | 63 | 1.6% | 76.2% |
| `tests20.dat` | General tree building | 64 | 0.0% | 85.9% |
| `tests21.dat` | Foreign content (SVG, MathML) | 23 | 0.0% | 91.3% |
| `tests22.dat` | Mis-nested formatting (adoption agency) | 5 | 0.0% | 0.0% |
| `tests23.dat` | Mis-nested formatting (adoption agency) | 5 | 0.0% | 0.0% |
| `tests24.dat` | Mis-nested formatting (adoption agency) | 8 | 0.0% | 0.0% |
| `tests25.dat` | Mis-nested formatting (adoption agency) | 26 | 0.0% | 76.9% |
| `tests26.dat` | Mis-nested formatting (adoption agency) | 20 | 0.0% | 5.0% |
| `tests3.dat` | General tree building | 24 | 16.7% | 58.3% |
| `tests4.dat` | Fragments and context | 9 | 33.3% | 44.4% |
| `tests5.dat` | Raw text, script data, unsafe text | 16 | 0.0% | 81.2% |
| `tests6.dat` | Fragments and context | 52 | 15.4% | 69.2% |
| `tests7.dat` | Tables and foster parenting | 34 | 0.0% | 58.8% |
| `tests8.dat` | Tables and foster parenting | 10 | 0.0% | 20.0% |
| `tests9.dat` | Foreign content (SVG, MathML) | 27 | 0.0% | 48.1% |
| `tests_innerHTML_1.dat` | Fragments and context | 81 | 25.9% | 42.0% |
| `tricky01.dat` | General tree building | 9 | 0.0% | 0.0% |
| `void-in-phrasing.dat` | General tree building | 13 | 0.0% | 100.0% |
| `webkit01.dat` | General tree building | 52 | 0.0% | 67.3% |
| `webkit02.dat` | General tree building | 48 | 0.0% | 50.0% |
To see the failures of one file or area:
`cargo run --release --manifest-path conformance/Cargo.toml -- --failures tables01.dat`.
The test data is © web-platform-tests contributors, under the
[3-Clause BSD License](https://github.com/web-platform-tests/wpt/blob/master/LICENSE.md);
it is downloaded by `conformance/fetch.sh`, not stored in this repository.