Crate parsoid[−][src]

Expand description

parsoid-rs

The parsoid crate is a wrapper around Parsoid HTML that provides convenient accessors for processing and extraction. Inspired by mwparserfromhell, parsoid-jsapi and built on top of Kuchiki (朽木).

Quick starts

Fetch HTML and extract the value of a template parameter:

use parsoid::prelude::*;
let client = ParsoidClient::new("https://en.wikipedia.org/api/rest_v1", "parsoid-rs demo")?;
let code = client.get("Taylor_Swift").await?;
for template in code.filter_templates()? {
    if template.name() == "Template:Infobox person" {
        let birth_name = template.get_param("birth_name").unwrap();
        assert_eq!(birth_name, "Taylor Alison Swift");
    }
}

Add a link to a page and convert it to wikitext:

use parsoid::prelude::*;
let client = ParsoidClient::new("https://en.wikipedia.org/api/rest_v1", "parsoid-rs demo")?;
let code = client.get("Wikipedia:Sandbox").await?;
let link = WikiLink::new(
    "./Special:Random",
    &Wikicode::new_text("Visit a random page")
);
code.append(&link);
let wikitext = client.transform_to_wikitext(&code).await?;
assert!(wikitext.ends_with("[[Special:Random|Visit a random page]]"));

This crate provides no functionality for actually saving a page, you’ll need to use something like mediawiki.

Architecture

Conceptually this crate provides wiki-related types on top of an HTML processing library. There are three primary constructs to be aware of: Wikicode, Wikinode, and Template.

Wikicode represents a container of an entire wiki page, equivalent to a <html> or <body> node. It some provides convenience functions like filter_links() to easily operate on and mutate a specific Wikinode. (For convenience, Wikicode is also a Wikinode.)

use parsoid::prelude::*;
let client = ParsoidClient::new("https://en.wikipedia.org/api/rest_v1", "parsoid-rs demo")?;
let code = client.get("Taylor Swift").await?;
for link in code.filter_links() {
    if link.target() == "You Belong with Me" {
        // ...do something
    }
}

Filter functions are only provided for common types as an optimization, but it’s straightforward to implement for other types:

use parsoid::prelude::*;
let client = ParsoidClient::new("https://en.wikipedia.org/api/rest_v1", "parsoid-rs demo")?;
let code = client.get("Taylor Swift").await?;
let entities: Vec<HtmlEntity> = code
    .descendants()
    .filter_map(|node| node.as_html_entity())
    .collect();

Wikinode is an enum representing all of the different types of Wikinodes, mostly to enable functions that accept/return various types of nodes.

A Wikinode provides convenience functions for working with specific types of MediaWiki constructs. For example, the WikiLink type wraps around a node of <a rel="mw:WikiLink" href="...">...</a>. It provides functions for accessing or mutating the href attribute. To access the link text you would need to use .children() and modify or append to those nodes. Standard mutators like .append() and .insert_after() are part of the WikinodeIterator trait, which is automatically imported in the prelude.

The following nodes have been implemented so far: