wikioxide

Parse MediaWiki wikitext into typed domain models — locally, deterministically, and without HTML: template calls with full parameter maps, categories with sort keys, internal/external links, typed parser warnings and structural statistics. A local replacement for action=parse with prop=templates|categories|links|externallinks|parsewarnings.

Beyond the API-parity extraction it also exposes a heading/section tree and a talk-page discussion parser (threaded comments with authors and timestamps) — both reported as byte spans into the original wikitext, so tooling can slice or move exact source.

Status: pre-release. Parse-only extraction, template expansion, the section tree, and the discussion parser all work behind a 100%-covered test suite; a real-article parity report and release polish are next.

Parse a page as written (no expansion):

use wikioxide::{DocumentView, PageContext, SiteConfig, WikitextParser};

let parser = WikitextParser::builder()
    .site_config(SiteConfig::ruwiki().clone())
    .build()?;
let doc = parser.parse("Стихи [[кота|котика]] в {{рамке|цвет=серый}}", &PageContext::default());
assert_eq!(doc.links()[0].target().text(), "Кота");
assert_eq!(doc.templates()[0].param("цвет"), Some("серый"));

Or expand templates first (post-expansion sets), fetching sources through a TemplateSource:

use std::sync::Arc;
use wikioxide::{
    DocumentView, ExpandOutcome, InMemoryTemplateStore, PageContext, SiteConfig, Title,
    WikitextParser,
};

let store = Arc::new(InMemoryTemplateStore::new());
store.insert(Title::parse("Template:Stub", SiteConfig::enwiki())?, "[[Category:Stubs]]");
let parser = WikitextParser::builder()
    .site_config(SiteConfig::enwiki().clone())
    .template_source(store)
    .build()?;
if let ExpandOutcome::Complete(doc) = parser.expand("Body.\n{{Stub}}", &PageContext::default()) {
    // The category comes from the template, post-expansion.
    assert_eq!(doc.categories()[0].category().text(), "Category:Stubs");
    assert_eq!(doc.transclusion_closure()[0].text(), "Template:Stub");
}

For an async caller, build with .on_unresolved(UnresolvedPolicy::Halt): expand then returns ExpandOutcome::MissingTemplates listing the pages to prefetch into the store before re-running.

Sections

ParsedDocument::sections() returns the nested heading tree. Every Section carries byte spans into the original source — the heading line, the inner title, and the section body (that heading through just before the next heading of the same or a higher level, subsections included). Body spans are source-contiguous even across templates and comments, so a consumer can slice or move a section verbatim.

use wikioxide::{PageContext, SiteConfig, WikitextParser};

let parser = WikitextParser::builder()
    .site_config(SiteConfig::enwiki().clone())
    .build()?;
let doc = parser.parse("== History ==\nText.\n=== Origins ===\nMore.", &PageContext::default());

let history = &doc.sections()[0];
assert_eq!(history.level(), 2);
assert_eq!(history.title_text(), "History");
assert_eq!(history.children()[0].title_text(), "Origins");

Talk-page discussions

Discussion::from_wikitext turns a saved talk page into signed, threaded comments — each with its author, timestamp, indentation level, and source spans. It works on saved wikitext (where ~~~~ is already substituted into literal [[User:…]] links and a (UTC) timestamp), so no expansion is needed; it is parameterised entirely by the SiteConfig (user namespaces and localized month names) and threads replies by leading :/*/# depth.

use wikioxide::SiteConfig;
use wikioxide::discussion::Discussion;

let src = "\
== Requested move ==
Support. [[User:Alice|Alice]] 08:26, 8 May 2026 (UTC)
: Agree. [[User:Bob|Bob]] 09:10, 8 May 2026 (UTC)
";
let discussion = Discussion::from_wikitext(src, SiteConfig::enwiki());

let reply = &discussion.comments()[1];
assert_eq!(reply.author().unwrap().name(), "Bob");
assert_eq!(reply.level(), 1);
assert_eq!(reply.parent(), Some(0)); // a reply to the nomination (comment 0)

Pair it with sections() to group comments per nomination: a comment belongs to a section when its span() falls within that section's body_span().

What "parity" means here

• templates / categories / links are post-expansion sets, like the API's: expand resolves templates (via a caller-supplied TemplateSource) before extracting. parse gives the as-written view instead.
• Supported parser functions: #if, #ifeq, #ifexpr, #iferror, #ifexist, #switch, #expr (full ExprParser port), #time, #tag, plus the string functions and the common magic words/variables. Unsupported ones are left literal with a warning.
• No Scribunto/Lua in v1: {{#invoke:}} records the Module: page in the closure, yields a typed invocation and a warning. Lua-heavy pages therefore under-expand.
• Warnings are a typed enum whose Display matches MediaWiki's parsewarnings strings; the parity ones report their message key via Warning::code().

Feature flags

The section tree and discussion parser are always available; they need no feature flag.

Development

./scripts/gate.sh      # fmt + clippy -D warnings + tests (both feature sets) + docs
cargo xtask coverage   # 100% line-coverage gate (nightly + cargo-llvm-cov)
cargo xtask gen-siteconfig ru.wikipedia.org   # regenerate a preset
cargo xtask fetch-parsertests                 # GPL test data → gitignored .parsertests/

MediaWiki's parser-test corpus is GPL-2.0+ and is therefore never committed here; the harness downloads it on demand and runs an allowlisted subset as data.

License

WTFPL — Do What The Fuck You Want To Public License, Version 2.