wikioxide 0.1.0

# wikioxide

Parse MediaWiki wikitext into **typed domain models** — locally, deterministically, and
without HTML: template calls with full parameter maps, categories with sort keys,
internal/external links, typed parser warnings and structural statistics. A local
replacement for `action=parse` with
`prop=templates|categories|links|externallinks|parsewarnings`.

Beyond the API-parity extraction it also exposes a **heading/section tree** and a
**talk-page discussion parser** (threaded comments with authors and timestamps) — both
reported as byte spans into the original wikitext, so tooling can slice or move exact source.

> Status: pre-release. Parse-only extraction, template expansion, the section tree, and the
> discussion parser all work behind a 100%-covered test suite; a real-article parity report
> and release polish are next.

Parse a page as written (no expansion):

```rust
use wikioxide::{DocumentView, PageContext, SiteConfig, WikitextParser};

let parser = WikitextParser::builder()
    .site_config(SiteConfig::ruwiki().clone())
    .build()?;
let doc = parser.parse("Стихи [[кота|котика]] в {{рамке|цвет=серый}}", &PageContext::default());
assert_eq!(doc.links()[0].target().text(), "Кота");
assert_eq!(doc.templates()[0].param("цвет"), Some("серый"));
```

Or expand templates first (post-expansion sets), fetching sources through a `TemplateSource`:

```rust
use std::sync::Arc;
use wikioxide::{
    DocumentView, ExpandOutcome, InMemoryTemplateStore, PageContext, SiteConfig, Title,
    WikitextParser,
};

let store = Arc::new(InMemoryTemplateStore::new());
store.insert(Title::parse("Template:Stub", SiteConfig::enwiki())?, "[[Category:Stubs]]");
let parser = WikitextParser::builder()
    .site_config(SiteConfig::enwiki().clone())
    .template_source(store)
    .build()?;
if let ExpandOutcome::Complete(doc) = parser.expand("Body.\n{{Stub}}", &PageContext::default()) {
    // The category comes from the template, post-expansion.
    assert_eq!(doc.categories()[0].category().text(), "Category:Stubs");
    assert_eq!(doc.transclusion_closure()[0].text(), "Template:Stub");
}
```

For an async caller, build with `.on_unresolved(UnresolvedPolicy::Halt)`: `expand` then returns
`ExpandOutcome::MissingTemplates` listing the pages to prefetch into the store before re-running.

## Sections

`ParsedDocument::sections()` returns the nested heading tree. Every `Section` carries byte
spans into the **original source** — the heading line, the inner title, and the section body
(that heading through just before the next heading of the same or a higher level, subsections
included). Body spans are source-contiguous even across templates and comments, so a consumer
can slice or move a section verbatim.

```rust
use wikioxide::{PageContext, SiteConfig, WikitextParser};

let parser = WikitextParser::builder()
    .site_config(SiteConfig::enwiki().clone())
    .build()?;
let doc = parser.parse("== History ==\nText.\n=== Origins ===\nMore.", &PageContext::default());

let history = &doc.sections()[0];
assert_eq!(history.level(), 2);
assert_eq!(history.title_text(), "History");
assert_eq!(history.children()[0].title_text(), "Origins");
```

## Talk-page discussions

`Discussion::from_wikitext` turns a saved talk page into signed, threaded comments — each with
its author, timestamp, indentation level, and source spans. It works on saved wikitext (where
`~~~~` is already substituted into literal `[[User:…]]` links and a `(UTC)` timestamp), so no
expansion is needed; it is parameterised entirely by the `SiteConfig` (user namespaces and
localized month names) and threads replies by leading `:`/`*`/`#` depth.

```rust
use wikioxide::SiteConfig;
use wikioxide::discussion::Discussion;

let src = "\
== Requested move ==
Support. [[User:Alice|Alice]] 08:26, 8 May 2026 (UTC)
: Agree. [[User:Bob|Bob]] 09:10, 8 May 2026 (UTC)
";
let discussion = Discussion::from_wikitext(src, SiteConfig::enwiki());

let reply = &discussion.comments()[1];
assert_eq!(reply.author().unwrap().name(), "Bob");
assert_eq!(reply.level(), 1);
assert_eq!(reply.parent(), Some(0)); // a reply to the nomination (comment 0)
```

Pair it with `sections()` to group comments per nomination: a comment belongs to a section
when its `span()` falls within that section's `body_span()`.

## What "parity" means here

- `templates` / `categories` / `links` are **post-expansion** sets, like the API's: `expand`
  resolves templates (via a caller-supplied `TemplateSource`) before extracting. `parse` gives
  the as-written view instead.
- Supported parser functions: `#if`, `#ifeq`, `#ifexpr`, `#iferror`, `#ifexist`, `#switch`,
  `#expr` (full ExprParser port), `#time`, `#tag`, plus the string functions and the common
  magic words/variables. Unsupported ones are left literal with a warning.
- **No Scribunto/Lua** in v1: `{{#invoke:}}` records the `Module:` page in the closure, yields a
  typed invocation and a warning. Lua-heavy pages therefore under-expand.
- Warnings are a typed enum whose `Display` matches MediaWiki's `parsewarnings` strings; the
  parity ones report their message key via `Warning::code()`.

## Feature flags

| Flag | Default | Effect |
|------|---------|--------|
| `presets` | ✓ | Vendored `SiteConfig::{enwiki, ruwiki, generic}()` + `from_preset_json` |
| `serde` | | `Serialize`/`Deserialize` on domain types |
| `parity` | | `formatversion=2` mirror types for API-shaped JSON |

The section tree and discussion parser are always available; they need no feature flag.

## Development

```sh
./scripts/gate.sh      # fmt + clippy -D warnings + tests (both feature sets) + docs
cargo xtask coverage   # 100% line-coverage gate (nightly + cargo-llvm-cov)
cargo xtask gen-siteconfig ru.wikipedia.org   # regenerate a preset
cargo xtask fetch-parsertests                 # GPL test data → gitignored .parsertests/
```

MediaWiki's parser-test corpus is GPL-2.0+ and is therefore **never committed** here; the
harness downloads it on demand and runs an allowlisted subset as data.

## License

[WTFPL](LICENSE) — Do What The Fuck You Want To Public License, Version 2.