CadForge

CadForge

CadForge discovers and downloads STEP CAD files from catalogue or detail pages. It resolves relative links, removes duplicates, downloads in bounded concurrent batches, and persists each batch without allowing remote filenames to escape the output directory.

use cadforge::{CadScraper, Config};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let scraper = CadScraper::new(Config { batch_size: 16, ..Default::default() })?;
    let files = scraper.scrape_pages(["https://example.com/models"]).await?;
    scraper.save_batch("./cad-data", &files)?;
    Ok(())
}

The crate follows ordinary HTML a[href] links and accepts .step, .stp, and .stepz paths. Site-specific APIs, JavaScript rendering, authentication, robots policy, rate limits, and terms remain the caller's responsibility.

License

MIT