Module spider::configuration

Configuration structure for Website.

Re-exports

• pub use crate::features::chrome_common::AuthChallengeResponse;
• pub use crate::features::chrome_common::AuthChallengeResponseResponse;
• pub use crate::features::chrome_common::CaptureScreenshotFormat;
• pub use crate::features::chrome_common::CaptureScreenshotParams;
• pub use crate::features::chrome_common::ClipViewport;
• pub use crate::features::chrome_common::ScreenShotConfig;
• pub use crate::features::chrome_common::ScreenshotParams;
• pub use crate::features::chrome_common::Viewport;
• pub use crate::features::chrome_common::WaitFor;
• pub use crate::features::chrome_common::WaitForDelay;
• pub use crate::features::chrome_common::WaitForIdleNetwork;
• pub use crate::features::chrome_common::WaitForSelector;
• pub use crate::features::openai_common::GPTConfigs;

Structs

• Configuration
  Structure to configure Website crawler

Enums

• RedirectPolicy
  Redirect policy configuration for request

Functions

• get_ua
  Get the user agent from the top agent list randomly.