Struct SmartKind 

pub struct SmartKind<P: BrowserProvider> { /* private fields */ }

HTTP-first execution with conservative browser promotion.

On the success path the detector inspects the response body and never promotes on status alone. On the error path HttpKind has already classified statuses such as 403 and 503 before an HTML context exists, so CrawlError::http_status is the only available signal. The default status list is deliberately small ([403, 503]): 429 means rate limiting, which a browser does not fix. The list is configurable, and sticky per-host promotion bounds repeated HTTP-first costs.

Implementations

impl<P: BrowserProvider> SmartKind<P>

pub fn builder(provider: P) -> SmartKindBuilder<P>

Starts configuring smart HTTP-first crawling backed by provider.

Trait Implementations

impl<P: BrowserProvider> CrawlerKind for SmartKind<P>

type Context = SmartContext

The context passed to user handlers and lifecycle hooks.

fn start<'a>( &'a self, env: &'a CrawlerEnv, ) -> BoxFuture<'a, Result<(), CrawlError>>

Runs once before the crawler fetches any request.

fn execute<'a>( &'a self, env: RequestEnv<'a>, ) -> BoxFuture<'a, Result<Self::Context, CrawlError>>

Executes one request attempt and constructs its handler context.

fn observe(&self, ctx: &Self::Context) -> AttemptObservation

Called once after execute() succeeds; feeds statistics, HandledRequest, and RetryStrategy metadata.

fn after_success<'a>( &'a self, ctx: &'a mut Self::Context, ) -> BoxFuture<'a, Result<(), CrawlError>>

Runs after the user handler succeeds.

fn cleanup( &self, outcome: RequestOutcome<Self::Context>, ) -> BoxFuture<'_, Result<(), CrawlError>>

Runs after every attempt concludes, regardless of its outcome.

fn stop<'a>( &'a self, env: &'a CrawlerEnv, ) -> BoxFuture<'a, Result<(), CrawlError>>

Runs once when crawler shutdown begins.

fn before_request<'a>( &'a self, prep: &'a mut RequestPrep, ) -> Pin<Box<dyn Future<Output = Result<(), CrawlError>> + Send + 'a>>

Mutates a request before an attempt executes.