helpprobe 0.1.0

# helpprobe

A comprehensive CLI tool discovery and automation framework that extracts structured information from command help text. Transform any CLI tool's help output into machine-readable data for automation, scripting, documentation generation, and IDE integration.

## Features

### 🔍 Command Discovery
- **Structured Parsing**: Extracts options, arguments, subcommands, and usage patterns from help text
- **Multi-Format Support**: Handles various help text formats (clap, GNU, custom)
- **Recursive Discovery**: Automatically discovers nested subcommands and builds complete command trees
- **Smart Caching**: Caches probe results for performance with automatic version detection
- **Automatic Help Flag Detection**: Intelligently tries common help flags (`--help`, `-h`, `help`, etc.) when none are provided

### 📊 Rich Metadata Extraction
- **Enhanced Options**: Extracts option types (boolean, string, number, choice), defaults, and descriptions
- **Argument Analysis**: Identifies required/optional arguments, types (path, URL, email, number), and variadic patterns
- **Environment Variables**: Discovers environment variable mappings and defaults
- **Validation Rules**: Extracts validation constraints (ranges, patterns, choices)
- **Examples**: Parses example commands from help text

### 🛠️ Code Generation
- **Shell Completion**: Generate completion scripts for Bash, Zsh, Fish, PowerShell, and NuShell
- **Command Builders**: Generate type-safe command builder code in Rust, Python, JavaScript, and TypeScript
- **API Documentation**: Generate documentation in Markdown, HTML, OpenAPI, and JSON Schema formats

### ✅ Validation
- **Command Validation**: Validate command invocations against discovered specifications
- **Type Checking**: Verify argument types and option values
- **Error Detection**: Identify missing required options/arguments, unknown flags, and type mismatches

## Installation

### From Source

```bash
git clone <repository-url>
cd helpprobe
cargo build --release
```

The binary will be available at `target/release/helpprobe`.

### Using Cargo

```bash
cargo install --path .
```

## Usage

### Basic Usage

Probe a command's help text. The program automatically detects and uses appropriate help flags:

```bash
helpprobe -- ls
helpprobe -- docker
helpprobe -- git
helpprobe -- cargo
```

You can also explicitly provide help flags:

```bash
helpprobe -- ls --help
helpprobe -- docker -h
```

### JSON Output

Get structured JSON output:

```bash
helpprobe --json -- ls --help
```

### Generate Shell Completion

Generate completion scripts that enable tab completion for commands in your shell. These scripts provide intelligent suggestions when you press Tab, showing available options, subcommands, and arguments.

**What are completion scripts?**
- They are shell-specific scripts (`.sh` for bash, `.ps1` for PowerShell, `.nu` for NuShell, etc.) that tell your shell how to complete commands
- When you type a command and press Tab, the script suggests valid completions (options, subcommands, arguments)
- Each shell (bash, zsh, fish, etc.) has its own format

**Example:** After loading a completion script for `docker`, typing `docker <TAB>` will show all available subcommands (build, run, ps, etc.), and `docker run --<TAB>` will show all available options.

**How to use them:**
1. **Generate** the script: `helpprobe --generate-completion <shell> --output <file> -- <command>`
2. **Load** it:
   - **Temporary** (current session only): `source <file>` (bash/zsh) or `. <file>` (PowerShell)
   - **Permanent** (all future sessions): Add it to your shell's configuration file (see examples below)
3. **Test**: Type the command and press Tab to see completions

The `--output` flag is recommended (works in all shells, prompts if file exists):

```bash
# Bash
# Step 1: Generate the completion script (creates docker-completion.sh)
helpprobe --generate-completion bash --output docker-completion.sh -- docker

# Step 2: Load it in your current session (enables tab completion)
source docker-completion.sh

# Step 3: Test it - type "docker <TAB>" to see subcommands, or "docker run --<TAB>" for options

# To make it permanent, add to ~/.bashrc:
# echo "source $(pwd)/docker-completion.sh" >> ~/.bashrc

# Zsh
helpprobe --generate-completion zsh --output _cargo -- cargo
# Add to fpath: mv _cargo ~/.zsh/completions/ && echo 'fpath=(~/.zsh/completions $fpath)' >> ~/.zshrc
# Then reload: source ~/.zshrc

# Fish
helpprobe --generate-completion fish --output ~/.config/fish/completions/git.fish -- git
# Fish automatically loads completions from ~/.config/fish/completions/

# PowerShell
helpprobe --generate-completion powershell --output podman-completion.ps1 -- podman
# Load in current session: . .\podman-completion.ps1
# Or add to PowerShell profile for persistence: Add-Content $PROFILE ". .\podman-completion.ps1"

# NuShell
helpprobe --generate-completion nushell --output ~/.config/nushell/cargo-completion.nu -- cargo
# Add to config.nu: echo 'source ~/.config/nushell/cargo-completion.nu' >> ~/.config/nushell/config.nu
```

**Note:** Instead of `--output`, you can use shell redirection: `helpprobe --generate-completion bash -- docker > docker-completion.sh` (in NuShell, use `| save` instead of `>`).
```

### Generate Command Builders

Generate type-safe command builder code that provides a fluent API for programmatically constructing CLI commands:

```bash
# Rust
helpprobe --generate-builder rust --output docker_builder.rs -- docker

# Python
helpprobe --generate-builder python --output git_builder.py -- git

# JavaScript/TypeScript
helpprobe --generate-builder typescript --output cargo_builder.ts -- cargo
```

**Use Case Example:**

Instead of manually constructing command strings, use the generated builder for type-safe, IDE-autocompleted command construction:

```python
# Before: Manual command construction (error-prone, no autocomplete)
import subprocess
subprocess.run(["docker", "run", "-d", "--name", "myapp", "-p", "8080:80", "nginx"])

# After: Using generated builder (type-safe, autocompleted)
from docker_builder import DockerBuilder

# Fluent API with autocomplete and type safety
result = DockerBuilder() \
    .run() \
    .detach() \
    .name("myapp") \
    .publish("8080:80") \
    .image("nginx") \
    .execute()

print(result.stdout)
```

```rust
// Rust example
use docker_builder::DockerBuilder;

let output = DockerBuilder::new()
    .run()
    .detach()
    .name("myapp")
    .publish("8080:80")
    .image("nginx")
    .execute()?;
```

This is especially useful for:
- **Automation scripts** - Build commands programmatically from configuration
- **Testing** - Construct test commands with compile-time safety
- **Wrappers** - Create higher-level APIs around CLI tools
- **Configuration management** - Generate commands from config files

### Generate API Documentation

Generate API documentation in various formats:

```bash
# Markdown
helpprobe --generate-api-docs markdown -- docker --help > docker.md

# HTML
helpprobe --generate-api-docs html -- git --help > git.html

# OpenAPI 3.0
helpprobe --generate-api-docs openapi -- cargo --help > cargo-openapi.json

# JSON Schema
helpprobe --generate-api-docs jsonschema -- kubectl --help > kubectl-schema.json
```

### Validate Commands

Validate command invocations:

```bash
helpprobe --validate -- docker run --image nginx
helpprobe --validate -- git commit --message "test"
```

### Recursive Discovery

Discover all subcommands recursively:

```bash
helpprobe --discover-all -- docker --help
helpprobe --discover-all --max-depth 3 -- git --help
```

### Caching

Cache is enabled by default. Control caching behavior:

```bash
# Disable cache
helpprobe --no-cache -- ls --help

# Clear cache for a command
helpprobe --clear-cache -- ls --help

# Use custom cache directory
helpprobe --cache-dir /tmp/my-cache -- docker --help
```

## Library Usage

Use `helpprobe` as a library in your Rust projects:

```rust
use helpprobe::{probe_command, ProbeConfig};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = ProbeConfig {
        timeout_secs: 3,
        require_help_flag: false,
        cache: None, // Or Some(cache::CacheConfig::default())
    };
    
    let args = vec!["--help".to_string()];
    let result = probe_command("ls", &args, &config).await?;
    
    println!("Found {} options", result.options.len());
    println!("Found {} subcommands", result.subcommands.len());
    println!("Found {} arguments", result.arguments.len());
    
    Ok(())
}
```

### Available Modules

- `helpprobe::parser` - Parsing functions for help text
- `helpprobe::completion` - Shell completion generation
- `helpprobe::builder` - Command builder code generation
- `helpprobe::api_docs` - API documentation generation
- `helpprobe::validation` - Command validation
- `helpprobe::cache` - Caching functionality

## Command-Line Options

```
USAGE:
    helpprobe [OPTIONS] -- <COMMAND> [ARGS...]

OPTIONS:
    --cache-dir <DIR>              Cache directory path (default: ~/.cache/helpprobe)
    --clear-cache                   Clear cache for the specified command
    --discover-all                  Recursively discover all subcommands
    --force                         Run command even when no help flag is present
    --generate-api-docs <FORMAT>    Generate API documentation (markdown, html, openapi, jsonschema)
    --generate-builder <LANGUAGE>   Generate command builder code (rust, python, javascript, typescript)
    --generate-completion <SHELL>   Generate shell completion script (bash, zsh, fish, powershell, nushell)
    --output <FILE>                 Output file for completion/builder/docs (prompts if exists, use --force to skip)
    --json                          Emit JSON instead of human-readable text
    --max-depth <DEPTH>             Maximum depth for recursive discovery (default: 5)
    --no-cache                      Disable caching of probe results
    --timeout-secs <SECS>           Timeout in seconds for the target command (default: 3)
    --validate                      Validate a command invocation
    --verbose                       Show raw stdout/stderr from the command
    -h, --help                      Print help information
    -V, --version                   Print version information
```

## Output Format

The JSON output includes:

- **Command Information**: Command name, arguments, exit code
- **Options**: Short/long flags, descriptions, types, defaults, choices
- **Arguments**: Names, types, required/optional, variadic, descriptions
- **Subcommands**: Names, descriptions, full paths, parent relationships
- **Environment Variables**: Names, descriptions, option mappings, defaults
- **Validation Rules**: Types, patterns, ranges, messages
- **Examples**: Command examples with descriptions and tags
- **Raw Output**: Original stdout/stderr for reference

## Use Cases

### Shell Completion Systems
Generate completion scripts for any CLI tool automatically.

### IDE Integration
Provide autocomplete and validation for CLI commands in IDEs.

### Testing Frameworks
Validate command invocations before execution in test suites.

### Documentation Generators
Auto-generate up-to-date documentation from help text.

### API Clients
Create type-safe wrappers around CLI tools.

### Configuration Management
Discover environment variables and generate config templates.

### CI/CD Pipelines
Validate and construct commands programmatically in automation scripts.

## Examples

### Example 1: Generate Bash Completion

```bash
helpprobe --generate-completion bash -- docker --help > /etc/bash_completion.d/docker
```

### Example 2: Validate Command in Script

```bash
if helpprobe --validate -- docker run --invalid-flag; then
    echo "Command is valid"
else
    echo "Command has errors"
    exit 1
fi
```

### Example 3: Generate Python Builder

```bash
helpprobe --generate-builder python -- git --help > git_builder.py

# Then use it:
python -c "
from git_builder import GitBuilder
cmd = GitBuilder().add().file('test.txt').build()
print(cmd)
"
```

### Example 4: Discover All Subcommands

```bash
helpprobe --discover-all --json -- docker --help | jq '.total_commands'
```

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for a detailed list of changes, new features, and bug fixes.

## Compatibility Specification

If you're developing a CLI tool and want to ensure optimal compatibility with `helpprobe`, see [HELP_PROBE_SPEC.md](HELP_PROBE_SPEC.md) for guidelines on formatting help output. This specification covers:

- Usage line formatting
- Options and flags formatting
- Subcommands formatting
- Arguments formatting
- Section headers and organization
- Best practices for maximum discoverability

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

Please make sure to:
- Add tests for new features
- Update documentation as needed
- Follow existing code style
- Run `cargo fmt` and `cargo clippy` before submitting

## License

This project is dual-licensed under:

- **MIT License** ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
- **Apache License, Version 2.0** ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)

You may choose either license for your use. This dual-licensing approach is common in the Rust ecosystem and provides maximum flexibility for users and contributors.

## Acknowledgments

Built with Rust for performance and reliability. Uses heuristic parsing to handle the wide variety of help text formats in the wild.