Expand description
Crawler lifecycle kinds, handles, and shared state. The crawler engine: lifecycle kinds, handles, and shared state.
Structs§
- Attempt
Observation - Metadata observed after a kind successfully constructs its handler context.
- Autoscaler
Snapshot - A point-in-time view of a crawler’s concurrency scaler.
- Basic
Context - The handler context for
BasicKind. - Basic
Kind - The no-fetch crawler kind whose execution hands requests directly to handlers.
- Crawler
- A configured crawler using lifecycle behavior supplied by
K. - Crawler
Builder - Builds a
Crawleraround a crawler kind. - Crawler
Env - Shared process-level state supplied to crawler lifecycle hooks.
- Crawler
Handle - A cheaply cloned weak back-reference to a running crawler.
- Request
Env - Per-attempt inputs supplied to
CrawlerKind::execute. - Request
Prep - Engine-owned scratch space passed to
CrawlerKind::before_request.
Enums§
- Crawler
Build Error - An error produced while building a crawler.
- Request
Outcome - The outcome supplied to per-attempt cleanup.
Traits§
- Crawler
Kind - Defines the complete lifecycle for one crawler flavor.
- Into
Start Request - A single value convertible into one start
Request. - Into
Start Requests - A collection of start requests accepted by
super::Crawler::run.
Type Aliases§
- Basic
Crawler - The no-HTTP crawler: drives the queue and hands requests straight to the handler.