Crate crates_llms_txt 

Crates LLMs TXT

A Rust library for extracting and processing Rust crate documentation from both online sources (docs.rs) and local crates. This library provides functionality to fetch, parse, and structure rustdoc JSON data for use in LLM training or other documentation processing tasks.

Features

• Online Documentation: Fetch documentation from docs.rs with automatic zstd decompression
• Local Documentation: Generate documentation from local Cargo projects
• Version Compatibility: Handle different rustdoc JSON format versions automatically
• Feature Control: Generate docs with specific feature sets or all features
• Flexible Output: Structured data suitable for LLM training or analysis

Examples

Fetching Online Documentation

use crates_llms_txt::CrateDocs;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Fetch latest version
    let docs = CrateDocs::from_online("serde", None).await?;
     
    // Fetch specific version
    let docs = CrateDocs::from_online("clap", Some("4.5.39".to_string())).await?;
     
    println!("Found {} documentation items", docs.sessions.len());
    Ok(())
}

Generating Local Documentation

use std::path::PathBuf;
use crates_llms_txt::CrateDocs;

// Generate docs with all features
let docs = CrateDocs::from_local(
    PathBuf::from("./Cargo.toml"),
    Some("stable".to_string()),
)?;

// Generate docs with specific features
let docs = CrateDocs::from_local_with_features(
    PathBuf::from("./Cargo.toml"),
    false, // don't disable default features
    Some(vec!["async".to_string()]),
    None, // auto-detect toolchain
)?;

Modules

error

Error Types and Result Definitions

fetch_docs

Online Documentation Fetcher

temp_trait

Temporary Trait Definitions and Compatibility Layer

Structs

CrateDocs

Main structure containing all documentation data for a crate.

FullSessionItem

Represents a full documentation session item with complete content.

SessionItem

Represents a single documentation session item with metadata.