siteone-crawler 2.5.0

Website crawler and QA toolkit in Rust for security, performance, SEO, and accessibility audits, offline cloning, markdown export, sitemap generation, cache warming, and CI/CD gating — one dependency-free binary for all major platforms, 10 tools in one.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

// SiteOne Crawler - AI subsystem
// (c) Jan Reges <jan.reges@siteone.cz>
//
// Optional AI features: provider-agnostic LLM client over the existing reqwest stack,
// per-page analyses (SEO, llms.txt, typos, custom prompt), with strict cost controls.
// Nothing here runs unless the user explicitly configures an AI provider.

pub mod actions;
pub mod client;
pub mod config;
pub mod normalize;
pub mod page;
pub mod prompt;
pub mod provider;
pub mod runner;
pub mod secret;
pub mod selection;
pub mod summary;
pub mod usage;