Trait CrawlerKind 

pub trait CrawlerKind:
    Send
    + Sync
    + 'static {
    type Context: Send + Clone + 'static;

    // Required methods
    fn execute<'a>(
        &'a self,
        env: RequestEnv<'a>,
    ) -> BoxFuture<'a, Result<Self::Context, CrawlError>>;
    fn cleanup(
        &self,
        outcome: RequestOutcome<Self::Context>,
    ) -> BoxFuture<'_, Result<(), CrawlError>>;

    // Provided methods
    fn start<'a>(
        &'a self,
        env: &'a CrawlerEnv,
    ) -> BoxFuture<'a, Result<(), CrawlError>> { ... }
    fn before_request<'a>(
        &'a self,
        prep: &'a mut RequestPrep,
    ) -> BoxFuture<'a, Result<(), CrawlError>> { ... }
    fn observe(&self, ctx: &Self::Context) -> AttemptObservation { ... }
    fn after_success<'a>(
        &'a self,
        ctx: &'a mut Self::Context,
    ) -> BoxFuture<'a, Result<(), CrawlError>> { ... }
    fn stop<'a>(
        &'a self,
        env: &'a CrawlerEnv,
    ) -> BoxFuture<'a, Result<(), CrawlError>> { ... }
}

Defines the complete lifecycle for one crawler flavor.

Required Associated Types

type Context: Send + Clone + 'static

The context passed to user handlers and lifecycle hooks.

A context must be a cheap aliasing handle over shared state, typically through Arc-backed fields. Clones must observe the same underlying resources so mutations through a handler’s clone remain visible to after_success and cleanup; plain-value contexts that diverge on clone violate this contract. Clone is required because the handler consumes an owned context while after_success and cleanup still need it.

Required Methods

fn execute<'a>( &'a self, env: RequestEnv<'a>, ) -> BoxFuture<'a, Result<Self::Context, CrawlError>>

Executes one request attempt and constructs its handler context.

fn cleanup( &self, outcome: RequestOutcome<Self::Context>, ) -> BoxFuture<'_, Result<(), CrawlError>>

Runs after every attempt concludes, regardless of its outcome.

Provided Methods

fn start<'a>( &'a self, env: &'a CrawlerEnv, ) -> BoxFuture<'a, Result<(), CrawlError>>

Runs once before the crawler fetches any request.

fn before_request<'a>( &'a self, prep: &'a mut RequestPrep, ) -> BoxFuture<'a, Result<(), CrawlError>>

Mutates a request before an attempt executes.

fn observe(&self, ctx: &Self::Context) -> AttemptObservation

Called once after execute() succeeds; feeds statistics, HandledRequest, and RetryStrategy metadata.

fn after_success<'a>( &'a self, ctx: &'a mut Self::Context, ) -> BoxFuture<'a, Result<(), CrawlError>>

Runs after the user handler succeeds.

fn stop<'a>( &'a self, env: &'a CrawlerEnv, ) -> BoxFuture<'a, Result<(), CrawlError>>

Runs once when crawler shutdown begins.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl CrawlerKind for BasicKind

type Context = BasicContext