opengrep 1.1.0

Advanced AST-aware code search tool with tree-sitter parsing and AI integration capabilities
Documentation

OpenGrep

Crates.io Downloads Rust License: MIT License: Apache 2.0 Docs.rs

Advanced AST-aware code search with AI-powered insights. OpenGrep understands your code structure and provides intelligent search capabilities beyond simple pattern matching.

Features

  • AST-Aware Search: Uses tree-sitter to understand code structure
  • Multi-Language Support: Supports 15+ programming languages
  • AI Integration: Optional OpenAI integration for intelligent insights
  • High Performance: Parallel search with efficient file traversal
  • Flexible Output: Text, JSON, HTML, XML, and CSV output formats
  • Interactive Mode: Built-in interactive search interface
  • Docker Ready: Containerized for easy deployment
  • Configurable: Extensive configuration options

Quick Start

Installation

From crates.io (Recommended)

cargo install opengrep

From Source

# Clone the repository
git clone https://github.com/opengrep-org/opengrep.git
cd opengrep

# Build and install
cargo install --path .

Using Docker

# Pull the image
docker pull opengrep/opengrep:latest

# Run a search
docker run --rm -v $(pwd):/workspace opengrep/opengrep "TODO" src/

Basic Usage

# Search for a pattern in current directory
opengrep "TODO" .

# Case-insensitive regex search
opengrep -i -e "fn\s+\w+" src/

# Show AST context around matches
opengrep -a "struct.*User" src/

# Interactive mode
opengrep -i

# JSON output
opengrep -o json "error" src/ > results.json

Supported Languages

  • Rust
  • Python
  • JavaScript/TypeScript
  • Go
  • Java
  • C/C++
  • C#
  • Ruby
  • Bash
  • JSON
  • TOML
  • CSS

Advanced Examples

AST-Aware Search

# Find all function definitions
opengrep -a "function_item" src/

# Search with AST context display
opengrep --ast-context --max-ast-depth 2 "impl" src/

AI-Powered Analysis

# Enable AI insights (requires OPENAI_API_KEY)
export OPENAI_API_KEY="your-api-key"
opengrep --ai-insights "memory leak" src/

# Get AI explanations for matches
opengrep --ai-explain "unsafe" src/

Output Formats

# HTML report
opengrep -o html "TODO" src/ > report.html

# CSV for spreadsheet analysis
opengrep -o csv "function" src/ > functions.csv

# Structured JSON
opengrep -o json --stats "error" src/

Filtering and Performance

# Filter by programming language
opengrep -l rust -l python "struct" .

# Exclude directories
opengrep -x "target/**" -x "node_modules/**" "TODO" .

# Limit file size and use more threads
opengrep --max-file-size 1048576 -j 8 "pattern" .

Configuration

OpenGrep can be configured via CLI arguments or configuration files:

Configuration File

Create ~/.config/opengrep/config.toml:

[search]
ignore_case = false
regex = false
threads = 8
max_file_size = 10485760

[output]
color = true
line_numbers = true
before_context = 2
after_context = 2
show_ast_context = false

[ai]
model = "gpt-4o-mini"
enable_insights = false
enable_explanation = false
max_tokens = 1000

Environment Variables

export OPENAI_API_KEY="your-openai-api-key"
export OPENGREP_CONFIG="/path/to/config.toml"
export RUST_LOG="info"  # Logging level

Building

Prerequisites

  • Rust 1.75+
  • Git

Development Build

./build.sh build

Release Build

./build.sh build 1.0.0

All Build Commands

./build.sh clean     # Clean previous builds
./build.sh format    # Format code
./build.sh lint      # Run linter
./build.sh test      # Run tests
./build.sh docker    # Build Docker image
./build.sh install   # Install locally

Docker Usage

Build Docker Image

docker build -t opengrep:latest .

Run with Docker

# Basic search
docker run --rm -v $(pwd):/workspace opengrep:latest "pattern" /workspace

# Interactive mode
docker run --rm -it -v $(pwd):/workspace opengrep:latest -i

# With AI features
docker run --rm -v $(pwd):/workspace -e OPENAI_API_KEY opengrep:latest --ai-insights "pattern" /workspace

Testing

# Run all tests
cargo test

# Run specific test module
cargo test ast::tests

# Run with output
cargo test -- --nocapture

# Integration tests
cargo test --test integration_tests

API Integration

OpenGrep can be used as a library:

use opengrep::{Config, SearchEngine};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = Config::default();
    let engine = SearchEngine::new(config);
    
    let results = engine.search("TODO", &[PathBuf::from("src")]).await?;
    
    for result in results {
        println!("Found {} matches in {}", 
                 result.matches.len(), 
                 result.path.display());
    }
    
    Ok(())
}

FastAPI Integration

For web service integration, see the examples directory for FastAPI endpoint implementations.

Performance

OpenGrep is optimized for performance:

  • Parallel Processing: Multi-threaded file traversal and searching
  • Smart Filtering: Respect .gitignore and file type filters
  • Memory Efficient: Streaming file processing
  • AST Caching: Intelligent AST parsing and caching

Benchmarks

# Run benchmarks
cargo bench

# Compare with other tools
hyperfine 'opengrep "pattern" .' 'rg "pattern" .' 'grep -r "pattern" .'

Troubleshooting

Common Issues

  1. AI features not working

    export OPENAI_API_KEY="your-api-key"

  2. Out of memory errors

    opengrep --max-file-size 1048576 "pattern" .

  3. Slow performance

    opengrep -j $(nproc) "pattern" .

  4. Language not detected

    opengrep --list-languages  # See supported languages

Debug Mode

RUST_LOG=debug opengrep "pattern" .

Contributing

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Development Setup

git clone https://github.com/opengrep-org/opengrep.git
cd opengrep
./build.sh format lint test

License

This project is dual-licensed under either:

at your option.

Acknowledgments

Support