Module spider::page

A page scraped.

Structs

• AIResults
  The AI data returned from a GPT.
• Page
  Represent a page visited. This page contains HTML scraped with scraper.

Functions

• build
  Instantiate a new page without scraping it (used for testing purposes).
• convert_abs_path
  convert to absolute path
• domain_name
  get the clean domain name
• get_page_selectors
  html selector for valid web pages for domain.
• parent_host_match
  validation to match a domain to parent host