pub struct Scrapers { /* private fields */ }Implementations§
Source§impl Scrapers
Interface to the collection of scrapers in this library.
impl Scrapers
Interface to the collection of scrapers in this library.
pub fn new(config: &ScrapeConfig) -> Self
Sourcepub fn compute_scrape_possibilities(&self) -> ScraperPossibilities
pub fn compute_scrape_possibilities(&self) -> ScraperPossibilities
Compute the list of all possible scrapes from all sources and subsources.
Sourcepub fn compute_scrape_subsources(&self, source: ScrapeSource) -> Vec<String>
pub fn compute_scrape_subsources(&self, source: ScrapeSource) -> Vec<String>
Compute the list of all possible scrapes from all sources and subsources.
Sourcepub fn compute_scrape_url_demands(
&self,
source: ScrapeSource,
subsources: Vec<String>,
) -> Vec<String>
pub fn compute_scrape_url_demands( &self, source: ScrapeSource, subsources: Vec<String>, ) -> Vec<String>
Given a source and subsources, compute the set of URLs to fetch.
Sourcepub fn scrape_http_result(
&self,
source: ScrapeSource,
input: ScraperHttpResponseInput,
) -> ScraperHttpResult
pub fn scrape_http_result( &self, source: ScrapeSource, input: ScraperHttpResponseInput, ) -> ScraperHttpResult
Given the result of fetching a URL, returns the scraped stories.
Auto Trait Implementations§
impl Freeze for Scrapers
impl RefUnwindSafe for Scrapers
impl Send for Scrapers
impl Sync for Scrapers
impl Unpin for Scrapers
impl UnwindSafe for Scrapers
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more