Struct FileStorage 

pub struct FileStorage { /* private fields */ }

Writes each fetch to a directory: base_dir/page/<fragment>.json (metadata) and base_dir/body/<fragment>.bin (raw body).

Implementations

impl FileStorage

pub fn new<P: AsRef<Path>>(base_path: P) -> Self

pub async fn ensure_dirs(&self) -> Result<()>

Trait Implementations

impl Clone for FileStorage

fn clone(&self) -> FileStorage

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Storage for FileStorage

fn record_fetch<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, job: &'life1 CrawlJob, result: &'life2 FetchResult, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Persist a fetch result. Called after each successful fetch.