Struct Context 

pub struct Context<C> { /* private fields */ }

A request being crawled, with everything needed to handle it.

It holds the Request, a session to fetch it with, a Queue to add the links it leads to, and the crawl’s datasets. Handlers borrow it, so several of them can handle one request: the request is fetched once, by the first response or body, and the complete response is kept. Its body is a fixed snapshot, including for browser sessions; live document operations go through the session.

The crawler learns how the fetch went, its time and status, from the task the request runs on. Fetching after moving the context to another task, e.g. with tokio::spawn, works, but adaptive host limits and Stats::unchanged do not see it.

Implementations

impl<C> Context<C>

pub fn new(request: Request, session: C) -> Self

Creates a standalone context, e.g. to test a handler.

It gets an empty queue of its own and no datasets; see with_queue and with_dataset.

pub fn with_queue(self, queue: &Queue) -> Self

Replaces the queue, binding it to the request.

pub fn with_dataset<T, D>(self, dataset: D) -> Self

where T: Send + 'static, D: Dataset<T>,

Registers dataset for items of type T.

pub fn with_extension<T: Clone + Send + Sync + 'static>(self, value: T) -> Self

Adds a service to this request, replacing any value of the same type.

pub const fn request(&self) -> &Request

Returns the request.

pub const fn request_mut(&mut self) -> &mut Request

Returns the request mutably, e.g. to add headers before fetching.

pub const fn session(&self) -> &C

Returns the session, including any live document capability.

pub const fn queue(&self) -> &Queue

Returns the queue for the links this request leads to.

pub fn dataset<T: Send + 'static>(&self) -> Result<Data<T>>

Returns the dataset for items of type T.

Errors

Returns a Dataset error if no dataset is registered for T.

pub async fn response(&self) -> Result<&Response>

where C: Session,

Returns the complete response, fetching the request on the first call.

Later calls return the same response without fetching again. A failed fetch is not kept: the next call tries again. Once fetched, links followed through the queue, including clones taken earlier, resolve against the URI the response was served from.

Errors

Returns an error if the session fails to fetch the request.

pub async fn body(&self) -> Result<Bytes>

where C: Session,

Returns a cheap clone of the cached response body, fetching first if needed. Browser sessions also return a fixed HTML snapshot; use the session’s document capability for live observations.

Errors

Returns an error if the request cannot be fetched.

Trait Implementations

impl<C> Debug for Context<C>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.