Trait Spider 

pub trait Spider:
    Send
    + Sync
    + 'static {
    type Item: ScrapedItem;

    // Required method
    fn parse<'life0, 'async_trait>(
        &'life0 mut self,
        response: Response,
    ) -> Pin<Box<dyn Future<Output = Result<ParseOutput<Self::Item>, SpiderError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided methods
    fn start_urls(&self) -> Vec<&'static str> { ... }
    fn start_requests(&self) -> Result<Vec<Request>, SpiderError> { ... }
}

Defines the contract for a web spider.

Required Associated Types

type Item: ScrapedItem

The type of item that the spider scrapes.

Required Methods

fn parse<'life0, 'async_trait>( &'life0 mut self, response: Response, ) -> Pin<Box<dyn Future<Output = Result<ParseOutput<Self::Item>, SpiderError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Parses a response and extracts scraped items and new requests.

Provided Methods

fn start_urls(&self) -> Vec<&'static str>

Returns the initial URLs to start crawling from.

fn start_requests(&self) -> Result<Vec<Request>, SpiderError>

Generates the initial requests to start crawling.

Implementors