Skip to main content

Module infrastructure

Module infrastructure 

Source
Expand description

Infrastructure layer โ€” External implementations (HTTP, FS, converters)

This layer contains the technical implementations of external concerns:

  • HTTP client creation
  • Web scraping (Readability, fallback)
  • Content conversion (HTML to Markdown, syntax highlighting)
  • File I/O (saving results, frontmatter generation)
  • Web crawling (FASE 1)
  • Export pipeline (JSONL, Zvec) (FASE 1)

Following Clean Architecture: infrastructure depends on domain, not vice versa.

Modulesยง

converter
Content converters (HTML โ†’ Markdown, syntax highlighting)
crawler
Crawler infrastructure module
export
Export pipeline implementations for RAG systems
http
HTTP client infrastructure
output
Output handling (file saving, frontmatter)
scraper
Scraping implementations