Struct url_scraper::UrlScraper

pub struct UrlScraper { /* fields omitted */ }

Stores the HTML document in memory.

Methods

impl UrlScraper

pub fn new(url: &str) -> Result<Self, Error>

Constructs a new scraper from a given URL.

pub fn new_with_client(url: &str, client: &Client) -> Result<Self, Error>

Use an existing reqwest::Client to make a request.

pub fn new_with_html(url: &str, html: &str) -> Result<Self, Error>

In case the HTML has already been fetched in advance, this can be used to parse from it directly.

Important traits for UrlIter<'a, 'b>

impl<'a, 'b> Iterator for UrlIter<'a, 'b> type Item = (String, Url);

pub fn into_iter<'a>(&'a self) -> UrlIter<'a, 'a>

Fetch the URLs using an iterator.