Trait scrappy_do::Handler[−][src]

pub trait Handler<I: Debug, C>: Send + Sync + Debug + Display {
    fn handle(
        self: Box<Self>, 
        client: Client, 
        response: Response, 
        context: C
    ) -> Receiver<Indeterminate<I, C>>;
}
Expand description

Converts HTTP responses into items or HTTP requests.

It is the job of the caller to define handlers. There are 2 primary methods of creating one. A conceptually stateless pure function method and a stateful struct implementation.

Pure function example

Pure function handlers should be your initial implementation and satisfy most needs. All you need to do is define the function and annotate it with handle:

// Needed for the yield keyword to work
#![feature(generators)]

use reqwest::{Client, Response};
use scrappy_do::handle;

// This is what we are trying to create from the web pages
#[derive(Debug)]
struct SomeItem;

// This tracks metadata we find useful during scraping.
#[derive(Debug)]
struct SomeContext;

#[handle(item = SomeItem)]
fn handler_foo(
    client: Client,
    response: Response,
    context: SomeContext,
) {

    // Process the response....

    // Return some scraped item
    yield SomeItem;
}

in order to get an actual Handler implementation you then just need to wrap the function using the provided macro:

ⓘ
wrap!(handler_foo)

Struct example

If your needs are more complex you may need to implement the trait yourself. Luckily the process isn’t much more complicated than the pure function method:

// Needed for the yield keyword to work
#![feature(generators)]

use reqwest::{Client, Response};
use scrappy_do::{Handler, handle};
use std::fmt;

// This is what we are trying to create from the web pages
#[derive(Debug)]
struct SomeItem;

// This tracks metadata we find useful during scraping.
#[derive(Debug)]
struct SomeContext;

#[derive(Debug)]
struct SomeHandler;

// Handlers need to implement Display
impl fmt::Display for SomeHandler {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "SomeHandler")
    }
}

impl Handler<SomeItem, SomeContext> for SomeHandler
{
    #[handle(item = SomeItem)]
    fn handle(self: Box<Self>,
              client: Client,
              response: Response,
              context: SomeContext) {

        // Process the response....

        // Return some scraped item
        yield SomeItem;
    }
}

Required methods

Implementors