Module web 

Web sanitize: URL fetch + 2-layer injection strip + envelope wrap. Covers the L4 × V1 cell (cf. crate root §Threat surface × vector).

Pipeline

fetch_and_sanitize composes:

1. fetch::get — HTTP GET with a 2 MiB body cap and a configurable timeout driven by crate::config::WebConfig::timeout_secs. Non-2xx responses and oversized bodies are rejected, not truncated.
2. sanitize::regex_layer::strip — fast local pass that replaces known injection markers with the literal [STRIPPED].
3. sanitize::semantic_layer::review — optional LLM-backed pass that runs only when ANTHROPIC_API_KEY is set (fail-open; see that module).
4. sanitize::wrap::envelope — wrap the cleaned body in an <untrusted_content src=... sanitized_at=... removed_patterns=...> envelope so the caller LLM physically separates instruction from data.

Result is persisted to <home>/web_log/<UTC-ts>-<id>.json as an audit row.

Read-only invariant

The fetched body is never written to disk in raw form. Only the sanitized envelope text and the list of removed patterns are persisted.

Modules

fetch

HTTP fetch with a hard body cap and a configurable timeout.

sanitize

Two-layer sanitize pipeline for fetched URL content.

Structs

SanitizedContent

One sanitize call’s full result payload.

Functions

fetch_and_sanitize

Fetch a URL, run the 2-layer sanitize pipeline, wrap in <untrusted_content>, and persist to <cfg.paths.home>/web_log/.