Module parser

cargo_docs_md

Module parser 

Source
Expand description

Rustdoc JSON parsing module.

This module handles loading and parsing rustdoc JSON files into the rustdoc_types::Crate structure that represents the entire documented crate.

§Rustdoc JSON Format

Rustdoc JSON is generated by running:

cargo doc --output-format json

The output is a single JSON file at target/doc/{crate_name}.json containing:

  • The crate’s module hierarchy
  • All public (and optionally private) items
  • Documentation strings
  • Type information and generics
  • Cross-reference links between items

§Key Types

The parsed Crate contains:

  • root: ID of the root module
  • index: HashMap of all items by their ID
  • paths: HashMap mapping IDs to their full module paths
  • crate_version: Optional version string

§Performance

When the simd-json feature is enabled, parsing uses SIMD-accelerated JSON parsing which is significantly faster for large rustdoc JSON files (10-50MB+). This requires AVX2/SSE4.2 on x86 platforms.

§Memory Usage

The entire rustdoc JSON file is loaded into memory and deserialized into a Crate structure. For typical crates (1-20MB JSON), this works well.

For very large crates (e.g., aws_sdk_ec2 at ~500MB), memory usage will be:

  • JSON file size in memory during parsing
  • Plus the deserialized Crate structure (usually similar size)
  • Peak memory ≈ 2x JSON file size

Future optimization: For extremely large crates, serde_json::StreamDeserializer could be used for incremental parsing, trading some simplicity for lower peak memory.

Structs§

Parser
Parser for rustdoc JSON files.