Expand description
Mapper: orchestrates the entire mapping process.
This is the core of the cartography engine. It uses a layered acquisition approach:
- Layer 0: Sitemap + robots.txt + HEAD scan + feed discovery
- Layer 1: HTTP GET sample pages + parse structured data (JSON-LD, OG, meta)
- Layer 1.5: Pattern engine (CSS selectors + regex) on pages with <50% structured data
- Layer 2: API discovery for known domains
- Layer 2.5: Action discovery — HTML forms + JS endpoints + platform templates
- Layer 3: Browser render ONLY for pages where Layers 0-2.5 gave <20% data
The browser is a last-resort fallback. For most e-commerce and news sites, Layers 1-2.5 provide sufficient data.
Structs§
- MapRequest
- Request to map a website.
- Mapper
- The Mapper orchestrates the entire site mapping process.