Skip to main content

HtmlKindBuilder

Struct HtmlKindBuilder 

Source
pub struct HtmlKindBuilder { /* private fields */ }
Expand description

Configures HtmlKind by delegating HTTP settings to HttpKindBuilder.

Implementations§

Source§

impl HtmlKindBuilder

Source

pub fn http_client(self, client: Arc<dyn HttpClient>) -> Self

Injects the HTTP transport.

Source

pub fn coalesce_in_flight(self, enabled: bool) -> Self

Enables or disables optional in-flight request coalescing. It is disabled by default.

Source

pub fn session_pool(self, options: SessionPoolOptions) -> Self

Enables sessions with the supplied pool options.

Source

pub fn shared_session_pool(self, pool: Arc<SessionPool>) -> Self

Uses an existing session pool without managing its persistence lifecycle.

Source

pub fn disable_sessions(self) -> Self

Disables session selection, cookie persistence, and session rotation.

Source

pub fn proxy(self, proxy: ProxyConfiguration) -> Self

Sets the default proxy bucket.

Source

pub fn proxy_buckets(self, proxies: ProxyBuckets) -> Self

Replaces all logical proxy buckets.

Source

pub fn proxy_strategy<S: ProxyStrategy>(self, strategy: S) -> Self

Sets the policy that selects a proxy bucket per attempt.

Source

pub fn user_agents<I, U>(self, user_agents: I) -> Self
where I: IntoIterator<Item = U>, U: Into<String>,

Replaces the rotating User-Agent set.

Source

pub fn retry_status_codes(self, codes: impl IntoIterator<Item = u16>) -> Self

Replaces the exact status codes classified as ordinary retries.

Source

pub fn retry_server_errors(self, enabled: bool) -> Self

Controls whether every 5xx response is retried.

Source

pub fn session_status_codes(self, codes: impl IntoIterator<Item = u16>) -> Self

Replaces statuses that trigger the retry_on_blocked session-rotation behavior.

Source

pub fn request_timeout(self, timeout: Duration) -> Self

Sets the HTTP request deadline.

Source

pub fn max_redirects(self, maximum: u32) -> Self

Sets the maximum number of redirects followed for one request.

Source

pub fn detect_anti_bot(self, detector: Arc<dyn AntiBotDetector>) -> Self

Enables response inspection with a caller-supplied anti-bot detector.

Source

pub fn detect_anti_bot_default(self) -> Self

Opts into the default anti-bot detector.

Source

pub fn header_generator(self, enabled: bool) -> Self

Enables or disables deterministic browser-like request headers.

Source

pub fn snapshot_errors_on_failure(self, enabled: bool) -> Self

Enables or disables response-body snapshots when handlers fail.

Source

pub fn pre_navigation_hook<F>(self, hook: F) -> Self
where F: for<'a> Fn(HttpPreHookCtx<'a>) -> BoxFuture<'a, Result<(), CrawlError>> + Send + Sync + 'static,

Registers an HTTP hook that runs immediately before navigation.

Source

pub fn post_navigation_hook<F>(self, hook: F) -> Self
where F: for<'a> Fn(HttpPostHookCtx<'a>) -> BoxFuture<'a, Result<(), CrawlError>> + Send + Sync + 'static,

Registers an HTTP hook that runs after navigation.

Source

pub fn build(self) -> Result<HtmlKind, HttpClientError>

Builds the kind, constructing the default HTTP client when none was injected.

Trait Implementations§

Source§

impl Default for HtmlKindBuilder

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more