Struct EnqueueLinksOptions 

#[non_exhaustive]
pub struct EnqueueLinksOptions<'a> { /* private fields */ }

Fluent options for one URLs-only enqueue operation.

Examples

use millipede_core::prelude::*;
use url::Url;

let result = EnqueueLinker::new(ctx.crawler.clone(), &ctx.request)
    .options()
    .raw_urls(["child", "/about"])
    .base_url(Url::parse("https://example.com/docs/")?)
    .label("detail")
    .limit(10)
    .send()
    .await?;
assert!(result.added.len() + result.skipped.len() <= 2);

Implementations

impl<'a> EnqueueLinksOptions<'a>

pub fn urls( self, urls: impl IntoIterator<Item = Url>, ) -> EnqueueLinksOptions<'a>

Extends the explicit absolute URL candidates.

These candidates bypass enqueue-strategy relationship filtering, including the default EnqueueStrategy::SameHostname. Other filters and crawl-policy limits still apply.

pub fn raw_urls<S>( self, urls: impl IntoIterator<Item = S>, ) -> EnqueueLinksOptions<'a>

where S: Into<String>,

Extends raw absolute or relative URL candidates.

pub fn base_url(self, base_url: Url) -> EnqueueLinksOptions<'a>

Overrides the base used to resolve raw relative URLs.

pub fn label(self, label: impl Into<String>) -> EnqueueLinksOptions<'a>

Applies a label to every child. Children do not inherit the parent label; without this option they remain unlabeled and use the router’s default route.

pub fn user_data(self, user_data: UserData) -> EnqueueLinksOptions<'a>

Applies user data to every child.

pub fn selector(self, selector: impl Into<String>) -> EnqueueLinksOptions<'a>

Selects links from the associated HTML context with a CSS selector.

pub fn strategy(self, strategy: EnqueueStrategy) -> EnqueueLinksOptions<'a>

Overrides the crawler policy’s URL relationship strategy.

pub fn globs<G>( self, globs: impl IntoIterator<Item = G>, ) -> EnqueueLinksOptions<'a>

where G: Into<GlobPattern>,

Adds URL include patterns. A candidate may match any glob or regex include.

pub fn regex( self, patterns: impl IntoIterator<Item = Regex>, ) -> EnqueueLinksOptions<'a>

Adds regular-expression URL includes.

pub fn exclude<P>( self, patterns: impl IntoIterator<Item = P>, ) -> EnqueueLinksOptions<'a>

where P: Into<UrlPattern>,

Adds URL exclusions. Exclusions take precedence over includes.

pub fn transform<F>(self, transform: F) -> EnqueueLinksOptions<'a>

where F: for<'r> Fn(&'r mut Request) -> Pin<Box<dyn Future<Output = TransformResult> + Send + 'r>> + Send + Sync + 'static,

Installs an asynchronous request transform that may mutate or reject each candidate.

pub fn limit(self, limit: usize) -> EnqueueLinksOptions<'a>

Caps the number of candidates after URL deduplication. Truncated candidates are silently omitted, and queue duplicates within the retained candidates consume the cap.

pub fn forefront(self, forefront: bool) -> EnqueueLinksOptions<'a>

Chooses whether children are inserted at the queue front.

pub async fn send(self) -> Result<EnqueueResult, CrawlError>

Resolves, builds, and enqueues candidates through one admission-controlled operation.