Struct MockSite 

pub struct MockSite { /* private fields */ }

Available on crate feature testing only.

A Backend serving fixed MockPages by URI, for tests.

URIs without a page get a 404 Not Found. Statuses are treated as the real backends treat them, by a StatusPolicy: by default, an error status fails the fetch, so the crawler retries or fails the request.

Clones share the record of requests.

Examples

use spire_core::Crawler;
use spire_core::frontier::seed::Seed;
use spire_core::testing::{MockPage, MockSite};
use spire_core::worker::from_fn;

let site = MockSite::new()
    .page("https://example.com/", MockPage::html("<p>Home</p>"))
    .page("https://example.com/old", MockPage::redirect("https://example.com/"));

let worker = from_fn(|cx: spire_core::Context<_>| async move { cx.body().await.map(drop) });
let crawler = Crawler::new(site.clone(), worker).with_seeds([
    Seed::get("https://example.com/old")?,
    Seed::get("https://example.com/missing")?,
]);

let stats = crawler.run().await?;
assert_eq!((stats.succeeded, stats.failed), (1, 1));
assert_eq!(site.requests().len(), 2);

Implementations

impl MockSite

pub fn new() -> Self

Creates a site without pages.

pub fn page(self, uri: &str, page: MockPage) -> Self

Serves page at uri, replacing any previous page there.

Panics

Panics if uri is not a valid absolute URI.

pub const fn with_status_policy(self, policy: StatusPolicy) -> Self

Replaces the StatusPolicy, Classify by default.

pub fn requests(&self) -> Vec<Uri>

Returns the URIs fetched so far, in order, including clones’ fetches. Redirects are followed within a fetch, so only the requested URI is listed.

pub fn visit(&self, seed: Seed) -> Visit

Prepares a Visit to seed, to run one request through a worker.

Trait Implementations

impl Backend for MockSite

type Client = MockClient

The client used to fetch a single request.

fn connect(&self) -> impl Future<Output = Result<Self::Client>> + Send

Returns a client for the next request.

impl Clone for MockSite

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MockSite

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MockSite

fn default() -> Self

Returns the “default value” for a type.