Enum CrawlError 

pub enum CrawlError {
    InvalidUrl(ParseError),
    RequestFailed(Error),
    TooManyRedirects(usize),
    BlockedByRobotsTxt(String),
    OutOfScope(String),
    Storage(String),
    MaxRetriesExceeded(usize),
}

Errors that can occur during crawl operations.

This is the primary error type for the crawlkit crate. It wraps lower-level errors from URL parsing, HTTP requests, robots.txt, and storage operations.

Examples

use crawlkit_engine::CrawlError;

let err = CrawlError::TooManyRedirects(20);
assert!(err.to_string().contains("20"));

Variants

InvalidUrl(ParseError)

The URL could not be parsed.

RequestFailed(Error)

The HTTP request failed.

TooManyRedirects(usize)

The URL exceeded the maximum redirect limit.

BlockedByRobotsTxt(String)

The URL is excluded by robots.txt disallow rules.

OutOfScope(String)

The URL is outside the allowed domain scope.

Storage(String)

A storage/database error occurred.

MaxRetriesExceeded(usize)

All retry attempts were exhausted.

Trait Implementations

impl Debug for CrawlError

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for CrawlError

fn fmt(&self, __formatter: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Error for CrawlError

fn source(&self) -> Option<&(dyn Error + 'static)>

Returns the lower-level source of this error, if any.

fn description(&self) -> &str

👎Deprecated since 1.42.0:

use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎Deprecated since 1.33.0:

replaced by Error::source, which can support downcasting

fn provide<'a>(&'a self, request: &mut Request<'a>)

🔬This is a nightly-only experimental API. (error_generic_member_access)

Provides type-based access to context intended for error reports.

impl From<CompareError> for CrawlError

fn from(e: CompareError) -> Self

Converts to this type from the input type.

impl From<Error> for CrawlError

fn from(source: Error) -> Self

Converts to this type from the input type.

impl From<ExportError> for CrawlError

fn from(e: ExportError) -> Self

Converts to this type from the input type.

impl From<ParseError> for CrawlError

fn from(source: ParseError) -> Self

Converts to this type from the input type.

impl From<StorageError> for CrawlError

fn from(e: StorageError) -> Self

Converts to this type from the input type.