pydocstring 0.4.0

# pydocstring

[![Crates.io Version](https://img.shields.io/crates/v/pydocstring?color=FFC12d)](https://crates.io/crates/pydocstring)
[![Crates.io MSRV](https://img.shields.io/crates/msrv/pydocstring?color=FFC12d)](https://blog.rust-lang.org/2025/02/20/Rust-1.85.0)
[![PyPI - Version](https://img.shields.io/pypi/v/pydocstring-rs?color=0062A8)](https://pypi.org/project/pydocstring-rs/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pydocstring-rs?color=0062A8)](https://devguide.python.org/versions/)

A zero-dependency Rust parser for Python docstrings (Google / NumPy style).

Produces a **unified syntax tree** with **byte-precise source locations** on every token — designed as infrastructure for linters and formatters.

Python bindings are also available as [`pydocstring-rs`](https://pypi.org/project/pydocstring-rs/).

## Features

- **Full syntax tree** — builds a complete AST, not just extracted fields; traverse it with the built-in `Visitor` + `walk_tree`
- **One code path for every style** — the unified view (`Document` → `Section` → `Entry`) reads any docstring with no style branching
- **Byte-precise source locations** — every token carries its exact byte range for pinpoint diagnostics
- **Zero dependencies** — pure Rust, no external crates, no regex
- **Error-resilient** — never panics; malformed input still yields a best-effort tree
- **Style auto-detection** — hand it a docstring, get back `Style::Google`, `Style::NumPy`, or `Style::Plain`
- **Pattern matching & rewriting** — find and `replace` constructs with `$NAME` / `$$$NAME` patterns, substituting byte-exact captured content so everything you don't rewrite is preserved

## Installation

```toml
[dependencies]
pydocstring = "0.4.0"
```

## Usage

### Parsing

Parse with auto-detection and traverse the style-independent views — one code
path for every docstring style:

```rust
use pydocstring::model::SectionKind;
use pydocstring::parse::{parse, Document};

let input = "Summary.\n\nArgs:\n    x (int): The value.\n    y (int): Another value.";
let parsed = parse(input);
let doc = Document::new(&parsed);

println!("{}", doc.summary().unwrap().text());

for section in doc.sections() {
    if section.kind() == SectionKind::Parameters {
        for entry in section.entries() {
            println!("{}: {}",
                entry.name().map(|n| n.text()).unwrap_or(""),
                entry.type_annotation().map(|t| t.text()).unwrap_or(""));
        }
    }
}
```

When you know the style (or want to force it), use the explicit parsers —
`parse_google` / `parse_numpy` / `parse_plain`. They all produce the same tree,
read through the same views.

### The Raw CST

The unified view is a *semantic* lens: it answers "is there a type?" and folds
away punctuation and the parser's zero-length placeholders. For the tree exactly
as parsed, go down to the CST with `syntax()`:

```rust
use pydocstring::parse::{parse, Document};
use pydocstring::syntax::SyntaxKind;

let parsed = parse("Summary.\n\nArgs:\n    x (): The value.\n");
let entry = Document::new(&parsed).sections().next().unwrap().entries().next().unwrap();

// The semantic lens says "no type" …
assert!(entry.type_annotation().is_none());
// … the CST says *why*: an empty type between brackets, whose zero-length range
// is the anchor to write one at. (`x:` — no brackets — has no placeholder.)
let placeholder = entry.syntax().find_missing(SyntaxKind::TYPE).unwrap();
assert!(placeholder.is_missing());
```

### Style Auto-Detection

```rust
use pydocstring::parse::{detect_style, Style};

assert_eq!(detect_style("Summary.\n\nArgs:\n    x: Desc."), Style::Google);
assert_eq!(detect_style("Summary.\n\nParameters\n----------\nx : int"), Style::NumPy);
assert_eq!(detect_style("Just a summary."), Style::Plain);
```

`Style::Plain` covers docstrings with no recognised section markers: summary-only,
summary + extended summary, and unrecognised styles such as Sphinx.

### Unified Auto-Detecting Parser

Use `parse()` to let the library detect the style and parse in one step:

```rust
use pydocstring::parse::{parse, Style};
use pydocstring::syntax::SyntaxKind;

let result = parse("Summary.\n\nArgs:\n    x: Desc.");
assert_eq!(result.root().kind(), SyntaxKind::DOCUMENT);
assert_eq!(result.style(), Style::Google);

let result = parse("Just a summary.");
assert_eq!(result.style(), Style::Plain);
```

### Source Locations

Every view and token carries byte offsets for precise diagnostics — and they
double as edit anchors:

```rust
use pydocstring::model::SectionKind;
use pydocstring::parse::{parse, Document};

let parsed = parse("Summary.\n\nArgs:\n    x (int): The value.");
let doc = Document::new(&parsed);

for section in doc.sections().filter(|s| s.kind() == SectionKind::Parameters) {
    for entry in section.entries() {
        let name = entry.name().unwrap();
        println!("'{}' at byte {}..{}", name.text(), name.range().start(), name.range().end());
    }
}
```

### Syntax Tree

The parse result is a tree of `SyntaxNode` (branches) and `SyntaxToken` (leaves), each tagged with a `SyntaxKind`. Use `pretty_print()` to visualize:

```rust
use pydocstring::parse::parse_google;

let result = parse_google("Summary.\n\nArgs:\n    x (int): The value.");
println!("{}", result.pretty_print());
```

```text
DOCUMENT@0..42 {
  SUMMARY: "Summary."@0..8
  SECTION@10..42 {
    SECTION_HEADER@10..15 {
      NAME: "Args"@10..14
      COLON: ":"@14..15
    }
    ENTRY@20..42 {
      NAME: "x"@20..21
      OPEN_BRACKET: "("@22..23
      TYPE: "int"@23..26
      CLOSE_BRACKET: ")"@26..27
      COLON: ":"@27..28
      DESCRIPTION: "The value."@29..39
    }
  }
}
```

### Visitor Pattern

Walk the tree with the `Visitor` trait for style-agnostic analysis:

```rust
use pydocstring::syntax::{Visitor, walk_tree, SyntaxToken, SyntaxKind};
use pydocstring::parse::parse_google;

struct NameCollector<'a> {
    source: &'a str,
    names: Vec<String>,
}

impl Visitor for NameCollector<'_> {
    fn visit_token(&mut self, token: &SyntaxToken) {
        if token.kind() == SyntaxKind::NAME {
            self.names.push(token.text(self.source).to_string());
        }
    }
}

let result = parse_google("Summary.\n\nArgs:\n    x: Desc.\n    y: Desc.");
let mut collector = NameCollector { source: result.source(), names: vec![] };
walk_tree(result.root(), &mut collector);
assert_eq!(collector.names, vec!["Args", "x", "y"]);
```

### Style-Independent Model (IR)

Convert any parsed docstring into a style-independent intermediate representation for analysis or transformation:

```rust
use pydocstring::model::SectionKind;
use pydocstring::parse::parse;

// `to_model()` dispatches on the parsed style, so this reads a NumPy
// docstring just as well.
let doc = parse("Summary.\n\nArgs:\n    x (int): The value.\n").to_model();

assert_eq!(doc.summary.as_deref(), Some("Summary."));
for section in &doc.sections {
    if section.kind == SectionKind::Parameters {
        // A section body is a flat sequence of blocks in source order.
        let params: Vec<_> = section.blocks.iter().filter_map(|b| b.as_parameter()).collect();
        assert_eq!(params[0].names, vec!["x"]);
        assert_eq!(params[0].type_annotation.as_deref(), Some("int"));
    }
}
```

### Emitting (Code Generation)

Re-emit a `Docstring` model in any style — useful for style conversion or formatting.
Google, NumPy, and Sphinx (reStructuredText) output are supported:

```rust
use pydocstring::model::{Docstring, Section, Parameter};
use pydocstring::emit::EmitOptions;
use pydocstring::emit::google::emit_google;
use pydocstring::emit::numpy::emit_numpy;
use pydocstring::emit::sphinx::emit_sphinx;

let doc = Docstring {
    summary: Some("Brief summary.".into()),
    sections: vec![Section::parameters(vec![Parameter {
        names: vec!["x".into()],
        type_annotation: Some("int".into()),
        description: Some("The value.".into()),
        is_optional: false,
        default_value: None,
    }])],
    ..Default::default()
};

let options = EmitOptions::default();
let google = emit_google(&doc, &options);
assert!(google.contains("Args:"));

let numpy = emit_numpy(&doc, &options);
assert!(numpy.contains("Parameters\n----------"));

let sphinx = emit_sphinx(&doc, &options);
assert!(sphinx.contains(":param x:"));
assert!(sphinx.contains(":type x: int"));

// Indent the output for embedding in a Python file:
let indented = emit_google(&doc, &EmitOptions::default().with_base_indent(4));
assert!(indented.starts_with("    Brief summary."));
```

> **Note:** Sphinx support is emit-only. `detect_style` reports Sphinx docstrings
> as `Style::Plain`, so parsing them yields a summary/extended-summary only.

Combine parsing and emitting to convert between styles:

```rust
use pydocstring::parse::parse_google;
use pydocstring::emit::EmitOptions;
use pydocstring::emit::numpy::emit_numpy;

let doc = parse_google("Summary.\n\nArgs:\n    x (int): The value.\n").to_model();
let numpy_text = emit_numpy(&doc, &EmitOptions::default());
assert!(numpy_text.contains("Parameters\n----------"));
```

### Pattern Matching & Rewriting

Find constructs with a `$NAME` / `$$$NAME` pattern and rewrite them with a
template. Captured metavariables substitute the **original source bytes**, so
everything you don't explicitly rewrite is preserved byte-for-byte — the issue
The issue #26 use case of annotating one entry without reflowing the rest:

```rust
use pydocstring::parse::{parse, Style};
use pydocstring::pattern::Pattern;

let src = "Summary.\n\nArgs:\n    x (int): The value.\n    y (str): Kept.\n";
let parsed = parse(src);
let pattern = Pattern::new(Style::Google, "$NAME (int): $DESC").unwrap();

let out = parsed.replace(&pattern, "$NAME (int): $DESC (deprecated)").unwrap();
assert_eq!(
    out,
    "Summary.\n\nArgs:\n    x (int): The value. (deprecated)\n    y (str): Kept.\n",
);
```

In Python the same lives on the docstring wrappers as `doc.replace(pattern, template)`
and `doc.findall(pattern)`:

```python
import pydocstring

doc = pydocstring.parse_google("Summary.\n\nArgs:\n    x (int): The value.\n")
new_src = doc.replace("$NAME ($TYPE): $DESC", "$NAME ($TYPE): $DESC (deprecated)")
for m in doc.findall("$NAME ($TYPE): $DESC"):
    print(m.capture("NAME").text, m.capture("TYPE").text)
```

## Supported Sections

Both styles support the same section categories. A section's role is read from
`Section::kind()` — `Args:` (Google) and `Parameters` (NumPy) both resolve to
`SectionKind::Parameters` — and every entry, whatever its role, is an `Entry`.

| Category                          | `SectionKind`                                        | Read with              |
|-----------------------------------|------------------------------------------------------|------------------------|
| Parameters                        | `Parameters`, `KeywordParameters`, `OtherParameters`, `Receives` | `section.entries()` |
| Returns / Yields                  | `Returns`, `Yields`                                  | `section.entries()`    |
| Raises / Warns                    | `Raises`, `Warns`                                    | `section.entries()`    |
| See Also                          | `SeeAlso`                                            | `section.entries()`    |
| Attributes / Methods              | `Attributes`, `Methods`                              | `section.entries()`    |
| References                        | `References`                                         | `section.citations()`  |
| Free text (Notes, Examples, …)    | `FreeText(_)`                                        | `section.body()`       |

Document-level: `summary()`, `extended_summary()`, `directives()` (a deprecation
notice is a directive named `deprecated`), and `paragraphs()`.

## Development

Common tasks are wrapped in a [`justfile`](justfile) — run `just` to list them:

```bash
just            # list all recipes
just lint       # cargo clippy (warnings as errors)
just test       # Rust test suite
just py-test    # build the Python extension (uv + maturin) and run pytest
just ci         # everything CI runs: fmt-check, lint, test, py-test
```

Or invoke cargo directly:

```bash
cargo build
cargo test
cargo run --example parse_auto
cargo run --example parse_google
cargo run --example parse_numpy
```