Skip to main content

Module worker

Module worker 

Source

Structs§

CrawlConfig

Functions§

run
Runs the crawl with the given frontier, seen set, storage, and rate limiter.
run_in_memory
In-memory backend for single-node runs.