Struct RefererMiddleware 

pub struct RefererMiddleware {
    pub same_origin_only: bool,
    pub max_chain_length: usize,
    pub include_fragment: bool,
    /* private fields */
}

Built-in middleware that is available without extra feature flags. Middleware that derives Referer values from request metadata and history.

Fields

same_origin_only: bool

Whether to use same-origin only referer

max_chain_length: usize

Maximum referer chain length to keep in memory

include_fragment: bool

Whether to include fragment in referer URL

Implementations

impl RefererMiddleware

pub fn new() -> RefererMiddleware

Creates a middleware with default settings.

pub fn same_origin_only(self, same_origin_only: bool) -> RefererMiddleware

Set whether to use same-origin only referer.

pub fn max_chain_length(self, max_chain_length: usize) -> RefererMiddleware

Set the maximum referer chain length to keep in memory.

pub fn include_fragment(self, include_fragment: bool) -> RefererMiddleware

Set whether to include the fragment in the referer URL.

Trait Implementations

impl Clone for RefererMiddleware

fn clone(&self) -> RefererMiddleware

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RefererMiddleware

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for RefererMiddleware

fn default() -> RefererMiddleware

Returns the “default value” for a type.

impl<C> Middleware<C> for RefererMiddleware

where C: Send + Sync,

fn name(&self) -> &str

Returns a human-readable middleware name for logs and diagnostics.

fn process_request<'life0, 'life1, 'async_trait>( &'life0 self, _client: &'life1 C, request: Request, ) -> Pin<Box<dyn Future<Output = Result<MiddlewareAction<Request>, SpiderError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, RefererMiddleware: 'async_trait,

Intercepts an outgoing request before the downloader runs.

fn process_response<'life0, 'async_trait>( &'life0 self, response: Response, ) -> Pin<Box<dyn Future<Output = Result<MiddlewareAction<Response>, SpiderError>> + Send + 'async_trait>>

where 'life0: 'async_trait, RefererMiddleware: 'async_trait,

Intercepts a successful response after download.

fn handle_error<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _request: &'life1 Request, error: &'life2 SpiderError, ) -> Pin<Box<dyn Future<Output = Result<MiddlewareAction<Request>, SpiderError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, Self: 'async_trait,

Handles downloader errors for a request.