Module scrape 

Web scraping executor with SSRF protection and domain policy enforcement.

Exposes two tools to the LLM:

• web_scrape — fetches a URL and extracts elements matching a CSS selector.
• fetch — fetches a URL and returns the raw response body as UTF-8 text.

Both tools enforce:

• HTTPS-only URLs (HTTP and other schemes are rejected).
• DNS resolution followed by a private-IP check to prevent SSRF.
• Optional domain allowlist and denylist from ScrapeConfig.
• Configurable timeout and maximum response body size.
• Redirect following is disabled to prevent open-redirect SSRF bypasses.

Structs

WebScrapeExecutor

Extracts data from web pages via CSS selectors.