github code searcher
A powerful, concurrent CLI tool for searching code across GitHub repositories with advanced rate-limit handling and progress visualization.
Features
- Efficient Code Searching: Search GitHub's codebase for specific keywords or phrases
- Concurrent Execution: Run multiple searches in parallel with configurable concurrency
- Smart Rate-Limit Handling: Automatically detects and waits for GitHub API rate limits with visual feedback
- Detailed Progress Tracking: Real-time progress indicators for each search term
- Pagination Support: Configure maximum pages per search term
- JSON Output: Structured output format for post-processing
Installation
Prerequisites
- Rust and Cargo installed (Install Rust)
- GitHub Personal Access Token (Create token)
Building from Source
# Clone the repository
# Build the project
# The binary will be available at
# ./target/release/github-code-searching
Usage
# Basic usage
# Search with environment variable for token
# Control concurrency and output location
# Limit pages per search term
Command-Line Arguments
Argument | Short | Long | Description | Default |
---|---|---|---|---|
Search terms | -w |
--words |
Words to search for (required, multiple allowed) | - |
Output file | -o |
--output |
Path for JSON results | search_results.json |
Max pages | -p |
--max-pages |
Maximum pages per search term | No limit |
GitHub token | -t |
--token |
GitHub API token | Uses GITHUB_TOKEN env var |
Concurrency | -c |
--concurrency |
Number of concurrent searches | 2 |
Authentication
The tool requires a GitHub token with appropriate permissions:
- Use the
--token
argument to provide your token directly - Set the
GITHUB_TOKEN
environment variable - Create a
.env
file in the project directory withGITHUB_TOKEN=your_token
Output Format
Results are saved in JSON format with the following structure for each match:
How It Works
GitHub Code Searcher performs the following operations:
- Parses command-line arguments and authenticates with GitHub
- Creates progress indicators for each search term
- Spawns concurrent search tasks limited by the specified concurrency
- Each search task handles pagination through search results
- Monitors and respects GitHub API rate limits with visual feedback
- Filters and saves relevant search results to the specified output file
Rate Limit Handling
The tool includes sophisticated rate-limit management:
- Monitors remaining API calls through GitHub response headers
- Displays real-time rate limit status
- Automatically pauses with countdown when limits are reached
- Resumes operations when rate limits reset
Development
Project Structure
src/main.rs
: Entry point and CLI argument parsingsrc/github_searcher.rs
: Core search functionality and API interaction
Building and Testing
# Run with debug output
RUST_LOG=debug
# Run tests
License
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.