pub struct CrawlResult {
pub stats: CrawlStats,
pub items: ItemList,
pub paused: bool,
}Expand description
The final output of a crawl run, bundling together statistics, scraped items, and a flag indicating whether the crawl ran to completion or was paused.
You obtain a CrawlResult by calling CrawlerEngine::crawl.
If paused is true, the engine saved a checkpoint to disk and you can
resume later by creating a new engine pointed at the same crawldir.
Fields§
§stats: CrawlStatsThe aggregate crawl statistics for the entire run, including request counts, byte totals, cache hit/miss ratios, and per-domain breakdowns.
items: ItemListThe collected scraped items. Use ItemList::to_json or
ItemList::to_jsonl to persist them to disk.
paused: boolWhether the crawl was paused (via CrawlerEngine::request_pause)
rather than completing naturally. When true, a checkpoint was saved and
the crawl can be resumed.
Implementations§
Auto Trait Implementations§
impl Freeze for CrawlResult
impl RefUnwindSafe for CrawlResult
impl Send for CrawlResult
impl Sync for CrawlResult
impl Unpin for CrawlResult
impl UnsafeUnpin for CrawlResult
impl UnwindSafe for CrawlResult
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more