Skip to main content

fetch_and_sanitize

Function fetch_and_sanitize 

Source
pub async fn fetch_and_sanitize(
    cfg: &Config,
    url: &str,
) -> Result<SanitizedContent>
Expand description

Fetch a URL, run the 2-layer sanitize pipeline, wrap in <untrusted_content>, and persist to <cfg.paths.home>/web_log/.

§Errors

The semantic sanitize layer fails open: a missing ANTHROPIC_API_KEY (i.e. cfg.llm.api_key == None) or non-2xx API response returns the regex-stripped text unchanged, so an LLM outage does not break URL fetch.