Struct CrawlerContext 

pub struct CrawlerContext<S, I>
where
    S: Spider<Item = I>,
    I: ScrapedItem,
{
    pub scheduler: Arc<Scheduler>,
    pub stats: Arc<StatCollector>,
    pub spider: Arc<S>,
    pub spider_state: Arc<S::State>,
    pub pipelines: Arc<Vec<Box<dyn Pipeline<I>>>>,
}

Aggregated context shared across all crawler tasks.

This struct bundles together all the Arc-wrapped components that need to be shared between the crawler’s various async tasks. Instead of cloning each Arc individually, we can clone this context with a single Arc::clone().

Fields

scheduler: Arc<Scheduler>

stats: Arc<StatCollector>

spider: Arc<S>

spider_state: Arc<S::State>

pipelines: Arc<Vec<Box<dyn Pipeline<I>>>>

Implementations

impl<S, I> CrawlerContext<S, I>

where S: Spider<Item = I>, I: ScrapedItem,

pub fn new( scheduler: Arc<Scheduler>, stats: Arc<StatCollector>, spider: Arc<S>, spider_state: Arc<S::State>, pipelines: Arc<Vec<Box<dyn Pipeline<I>>>>, ) -> Self

Creates a new CrawlerContext with the given components.

pub fn from_crawler<C>( scheduler: Arc<Scheduler>, stats: Arc<StatCollector>, spider: Arc<S>, spider_state: Arc<S::State>, pipelines: Arc<Vec<Box<dyn Pipeline<I>>>>, ) -> Self

Creates a CrawlerContext from a Crawler instance.

Trait Implementations

impl<S, I> Clone for CrawlerContext<S, I>

where S: Spider<Item = I>, I: ScrapedItem,

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.