Skip to main content

Module mapper

Module mapper 

Source
Expand description

Mapper: orchestrates the entire mapping process.

This is the core of the cartography engine. It uses a layered acquisition approach:

  1. Layer 0: Sitemap + robots.txt + HEAD scan + feed discovery
  2. Layer 1: HTTP GET sample pages + parse structured data (JSON-LD, OG, meta)
  3. Layer 1.5: Pattern engine (CSS selectors + regex) on pages with <50% structured data
  4. Layer 2: API discovery for known domains
  5. Layer 2.5: Action discovery — HTML forms + JS endpoints + platform templates
  6. Layer 3: Browser render ONLY for pages where Layers 0-2.5 gave <20% data

The browser is a last-resort fallback. For most e-commerce and news sites, Layers 1-2.5 provide sufficient data.

Structs§

MapRequest
Request to map a website.
Mapper
The Mapper orchestrates the entire site mapping process.