pub struct BrowserKind<P: BrowserProvider> { /* private fields */ }Expand description
Browser fetching behavior used by BrowserCrawler.
Session persistence has explicit ownership semantics. An owned pool is attached to crawler
storage, restored in CrawlerKind::start, persisted on PersistState, and persisted once
more during CrawlerKind::stop. A shared pool is used for session selection and cookie
synchronization only: the sharing owner (for example, a smart kind) must restore and persist
it. This prevents double restore and double persistence when kinds share one pool.
Implementations§
Source§impl<P: BrowserProvider> BrowserKind<P>
impl<P: BrowserProvider> BrowserKind<P>
Sourcepub fn builder(provider: P) -> BrowserKindBuilder<P>
pub fn builder(provider: P) -> BrowserKindBuilder<P>
Starts configuring a browser crawler kind backed by provider.
Trait Implementations§
Source§impl<P: BrowserProvider> CrawlerKind for BrowserKind<P>
impl<P: BrowserProvider> CrawlerKind for BrowserKind<P>
Source§type Context = BrowserContext
type Context = BrowserContext
The context passed to user handlers and lifecycle hooks. Read more
Source§fn start<'a>(
&'a self,
env: &'a CrawlerEnv,
) -> BoxFuture<'a, Result<(), CrawlError>>
fn start<'a>( &'a self, env: &'a CrawlerEnv, ) -> BoxFuture<'a, Result<(), CrawlError>>
Runs once before the crawler fetches any request.
Source§fn execute<'a>(
&'a self,
env: RequestEnv<'a>,
) -> BoxFuture<'a, Result<Self::Context, CrawlError>>
fn execute<'a>( &'a self, env: RequestEnv<'a>, ) -> BoxFuture<'a, Result<Self::Context, CrawlError>>
Executes one request attempt and constructs its handler context.
Source§fn observe(&self, ctx: &Self::Context) -> AttemptObservation
fn observe(&self, ctx: &Self::Context) -> AttemptObservation
Called once after
execute() succeeds; feeds statistics, HandledRequest, and
RetryStrategy metadata.Source§fn cleanup(
&self,
outcome: RequestOutcome<Self::Context>,
) -> BoxFuture<'_, Result<(), CrawlError>>
fn cleanup( &self, outcome: RequestOutcome<Self::Context>, ) -> BoxFuture<'_, Result<(), CrawlError>>
Runs after every attempt concludes, regardless of its outcome.
Source§fn stop<'a>(
&'a self,
_env: &'a CrawlerEnv,
) -> BoxFuture<'a, Result<(), CrawlError>>
fn stop<'a>( &'a self, _env: &'a CrawlerEnv, ) -> BoxFuture<'a, Result<(), CrawlError>>
Runs once when crawler shutdown begins.
Source§fn before_request<'a>(
&'a self,
prep: &'a mut RequestPrep,
) -> Pin<Box<dyn Future<Output = Result<(), CrawlError>> + Send + 'a>>
fn before_request<'a>( &'a self, prep: &'a mut RequestPrep, ) -> Pin<Box<dyn Future<Output = Result<(), CrawlError>> + Send + 'a>>
Mutates a request before an attempt executes.
Auto Trait Implementations§
impl<P> !Freeze for BrowserKind<P>
impl<P> !RefUnwindSafe for BrowserKind<P>
impl<P> !UnwindSafe for BrowserKind<P>
impl<P> Send for BrowserKind<P>
impl<P> Sync for BrowserKind<P>
impl<P> Unpin for BrowserKind<P>
impl<P> UnsafeUnpin for BrowserKind<P>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more