Crate opengrep

Expand description

§OpenGrep Library

Advanced AST-aware code search with AI-powered insights. OpenGrep understands code structure and provides intelligent search capabilities beyond simple pattern matching.

§Overview

OpenGrep is a powerful code search tool that combines traditional pattern matching with Abstract Syntax Tree (AST) analysis and optional AI integration. It supports over 15 programming languages and provides various output formats for integration into development workflows.

§Key Features

AST-Aware Search: Uses tree-sitter to understand code structure and context
Multi-Language Support: Supports 20+ programming languages out of the box
AI Integration: Optional OpenAI integration for intelligent code insights
High Performance: Parallel search with efficient file traversal and caching
Flexible Output: Multiple output formats (Text, JSON, HTML, XML, CSV)
Interactive Mode: Built-in interactive search interface with fuzzy matching
Configuration: Extensive configuration options via files and environment variables

§Supported Languages

System Languages: Rust, C, C++, Go
Web Languages: JavaScript, TypeScript, CSS, HTML
Enterprise Languages: Java, C#, Python
Scripting Languages: Ruby, Bash, Shell
Data Languages: JSON, TOML, YAML, XML

§Quick Start

§Basic Search

use opengrep::{Config, SearchEngine};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Create default configuration
    let config = Config::default();
    let engine = SearchEngine::new(config);
     
    // Search for TODO comments in source directory
    let results = engine.search("TODO", &[PathBuf::from("src")]).await?;
     
    // Process results
    for result in results {
        println!("Found {} matches in {}", 
                 result.matches.len(), 
                 result.path.display());
         
        for match_item in &result.matches {
            println!("  Line {}: {}", 
                     match_item.line_number, 
                     match_item.line_text.trim());
        }
    }
     
    Ok(())
}

§AST-Aware Search

use opengrep::{Config, SearchEngine};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let mut config = Config::default();
    config.output.show_ast_context = true;
    config.output.max_ast_depth = 3;
     
    let engine = SearchEngine::new(config);
     
    // Search for function implementations with AST context
    let results = engine.search("fn.*impl", &[PathBuf::from("src")]).await?;
     
    for result in results {
        for match_item in &result.matches {
            if let Some(ast_context) = &match_item.ast_context {
                println!("Found in {}: {}", 
                         ast_context.summary, 
                         match_item.line_text.trim());
            }
        }
    }
     
    Ok(())
}

§Custom Configuration

use opengrep::{Config, SearchConfig, OutputConfig, AIConfig};

let mut config = Config::default();
 
// Configure search behavior
config.search.ignore_case = true;
config.search.regex = true;
config.search.context = 3;
config.search.threads = 8;
config.search.max_file_size = 10 * 1024 * 1024; // 10MB
 
// Configure output formatting
config.output.show_line_numbers = true;
config.output.show_ast_context = true;
config.output.color = true;
config.output.before_context = 2;
config.output.after_context = 2;
 
// Configure AI features (optional)
#[cfg(feature = "ai")]
{
    config.ai.model = "gpt-4o-mini".to_string();
    config.ai.enable_insights = true;
    config.ai.max_tokens = 2000;
}

§Parallel Processing

use opengrep::{Config, SearchEngine};
use std::path::PathBuf;
 
#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let mut config = Config::default();
    config.search.threads = num_cpus::get(); // Use all available cores
     
    let engine = SearchEngine::new(config);
     
    // Search multiple directories in parallel
    let paths = vec![
        PathBuf::from("src"),
        PathBuf::from("tests"),
        PathBuf::from("examples"),
    ];
     
    let results = engine.search("FIXME|TODO|XXX", &paths).await?;
     
    println!("Found {} files with issues", 
             results.iter().filter(|r| !r.matches.is_empty()).count());
     
    Ok(())
}

§Output Formats

OpenGrep supports multiple output formats for integration with different tools:

§JSON Output

use opengrep::{Config, SearchEngine, output::OutputFormatter};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = Config::default();
    let engine = SearchEngine::new(config);
    let results = engine.search("pattern", &[PathBuf::from(".")]).await?;
     
    // Output as JSON
    let json_output = serde_json::to_string_pretty(&results)?;
    println!("{}", json_output);
     
    Ok(())
}

§Error Handling

OpenGrep uses the anyhow crate for error handling, providing rich error context:

use opengrep::{Config, SearchEngine};
use std::path::PathBuf;

#[tokio::main]
async fn main() {
    let config = Config::default();
    let engine = SearchEngine::new(config);
     
    match engine.search("pattern", &[PathBuf::from("nonexistent")]).await {
        Ok(results) => {
            println!("Found {} results", results.len());
        }
        Err(e) => {
            eprintln!("Search failed: {}", e);
            // Error context includes file paths, parsing errors, etc.
            for cause in e.chain().skip(1) {
                eprintln!("  Caused by: {}", cause);
            }
        }
    }
}

§Performance Optimization

For large codebases, consider these performance optimizations:

use opengrep::{Config, SearchConfig};

let mut config = Config::default();

// Increase thread count for I/O bound operations
config.search.threads = num_cpus::get() * 2;

// Set reasonable file size limits
config.search.max_file_size = 5 * 1024 * 1024; // 5MB limit

// Use ignore patterns to skip irrelevant files
config.search.ignore_patterns = vec![
    "*.log".to_string(),
    "target/**".to_string(),
    "node_modules/**".to_string(),
];

// Enable memory-efficient streaming for large files
config.search.streaming = true;

§AI Integration

When the ai feature is enabled, OpenGrep can provide intelligent code insights:

#[cfg(feature = "ai")]
use opengrep::{Config, SearchEngine, ai::AIAnalyzer};

#[cfg(feature = "ai")]
#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let mut config = Config::default();
    config.ai.enable_insights = true;
    config.ai.model = "gpt-4o-mini".to_string();
     
    let engine = SearchEngine::new(config);
    let results = engine.search("unsafe", &[std::path::PathBuf::from("src")]).await?;
     
    // AI analysis provides context about unsafe code usage
    for result in results {
        for match_item in &result.matches {
            if let Some(ai_insight) = &match_item.ai_insight {
                println!("AI Analysis: {}", ai_insight.summary);
                println!("Suggestion: {}", ai_insight.suggestion);
            }
        }
    }
     
    Ok(())
}

§Integration Examples

§CI/CD Pipeline Integration

use opengrep::{Config, SearchEngine};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = Config::default();
    let engine = SearchEngine::new(config);
     
    // Check for common issues in CI
    let issues = vec![
        ("TODO", "Unfinished work"),
        ("FIXME", "Known bugs"),
        ("XXX", "Problematic areas"),
        ("println!", "Debug statements"),
    ];
     
    let mut total_issues = 0;
     
    for (pattern, description) in issues {
        let results = engine.search(pattern, &[PathBuf::from("src")]).await?;
        let count: usize = results.iter().map(|r| r.matches.len()).sum();
         
        if count > 0 {
            println!("Warning: Found {} instances of {}", count, description);
            total_issues += count;
        }
    }
     
    if total_issues > 0 {
        std::process::exit(1); // Fail CI if issues found
    }
     
    Ok(())
}

§Code Review Integration

use opengrep::{Config, SearchEngine, output::json::JsonFormatter};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let mut config = Config::default();
    config.output.show_ast_context = true;
     
    let engine = SearchEngine::new(config);
     
    // Find potential security issues
    let security_patterns = vec![
        "unsafe",
        "transmute",
        "eval\\(",
        "exec\\(",
        "system\\(",
    ];
     
    for pattern in security_patterns {
        let results = engine.search(pattern, &[PathBuf::from(".")]).await?;
         
        if !results.is_empty() {
            // Generate detailed JSON report for code review tools
            let json_report = serde_json::to_string_pretty(&results)?;
            std::fs::write(format!("security-report-{}.json", pattern), json_report)?;
        }
    }
     
    Ok(())
}

Re-exports§

pub use config::Config;
pub use search::SearchEngine;
pub use search::SearchResult;
pub use search::Match;

Modules§

ast: Abstract Syntax Tree analysis and context extraction
cli: Command-line interface for OpenGrep
config: Configuration structures for OpenGrep
output: Output formatting and rendering
parsers: Language detection and parser management
prelude: Re-export commonly used types for convenience Common types and traits for typical usage
search: Core search functionality

Constants§

VERSION: Library version

Functions§

init_logging: Initialize logging based on verbosity level

Crate opengrepCopy item path