Trait CrawlSink 

pub trait CrawlSink: Send + Sync {
    // Required methods
    fn page(&self, doc: Document) -> BoxFuture<'_, ()>;
    fn error<'a>(&'a self, url: &'a str, err: ApiError) -> BoxFuture<'a, ()>;
    fn total(&self, total: usize) -> BoxFuture<'_, ()>;
    fn cancelled(&self) -> BoxFuture<'_, bool>;

    // Provided methods
    fn resume(&self) -> Option<CrawlResume> { ... }
    fn tracks_frontier(&self) -> bool { ... }
    fn enqueued(&self, _urls: Vec<(String, usize)>) -> BoxFuture<'_, ()> { ... }
    fn summary(&self, _summary: CrawlSummary) -> BoxFuture<'_, ()> { ... }
}

整站爬取的进度回报，由 API 层的任务管理实现。

Required Methods

fn page(&self, doc: Document) -> BoxFuture<'_, ()>

fn error<'a>(&'a self, url: &'a str, err: ApiError) -> BoxFuture<'a, ()>

fn total(&self, total: usize) -> BoxFuture<'_, ()>

已经确定要抓的页面总数（随着发现新链接增长）。

fn cancelled(&self) -> BoxFuture<'_, bool>

Provided Methods

fn resume(&self) -> Option<CrawlResume>

断点续爬：任务是从中断处恢复的时候返回上次的进度，引擎据此重建队列和去重表， 不再抓已经完成的页面。新任务返回 None。

fn tracks_frontier(&self) -> bool

是否需要 CrawlSink::enqueued 回报（持久化的任务需要）。不需要时引擎不必收集。

fn enqueued(&self, _urls: Vec<(String, usize)>) -> BoxFuture<'_, ()>

新进入队列的 URL 和发现深度，按入队顺序。只在 CrawlSink::tracks_frontier 为真时调用。

fn summary(&self, _summary: CrawlSummary) -> BoxFuture<'_, ()>

爬取结束时的说明（停止原因等）；只有带查询的爬取会调用。

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors