Crate webpage 

Small library to fetch info about a web page: title, description, language, HTTP info, links, RSS feeds, Opengraph, Schema.org, and more

Usage

use webpage::{Webpage, WebpageOptions};

let info = Webpage::from_url("http://example.org", WebpageOptions::default())
    .expect("Could not read from URL");

// the HTTP transfer info
println!("Effective URL: {}", info.http.url);
println!("Content type: {}", info.http.content_type);

// the parsed HTML info
println!("Title: {:?}", info.html.title);
println!("Links: {}", info.html.links.len());

You can also get HTML info about local data:

use webpage::HTML;
let html = HTML::from_file("index.html", None);
// or let html = HTML::from_string(input, None);

Options

The following configurations are available:

pub struct WebpageOptions {
    allow_insecure: bool,
    follow_location: bool,
    max_redirections: u32,
    timeout: std::time::Duration,
    useragent: String,
    headers: Vec<String>,
}

use webpage::{Webpage, WebpageOptions};

let mut options = WebpageOptions::default();
options.allow_insecure = true;
let info = Webpage::from_url("https://example.org", options).expect("Halp, could not fetch");

Structs

HTML

Information regarding the HTML content

HTTP

Information regarding the HTTP transfer

Link

Information for an <a> anchor

Opengraph

Representing OpenGraph information

OpengraphObject

Info about an OpenGraph media type

SchemaOrg

Representing Schema.org information (currently only via JSON-LD)

Webpage

All gathered info for a webpage

WebpageOptions

Configuration options for fetching a webpage