Expand description
§duckduckgo-search-cli
Rust CLI for searching DuckDuckGo via pure HTTP, with structured JSON output for LLM consumption. No paid API. No Chrome (during the search phase). No cache. Universal cross-platform (Linux including Alpine/NixOS/Flatpak/Snap, macOS including Apple Silicon, Windows including cmd.exe and PowerShell).
§Module Structure
| Module | Responsibility |
|---|---|
cli | Clap structs (command-line argument parsing). |
http | reqwest::Client construction and User-Agent selection. |
search | URL building and HTTP request to the DuckDuckGo endpoint. |
extraction | HTML parsing with scraper and ad filtering. |
pipeline | Single/multi orchestration, deduplication and source reading. |
parallel | Multi-query fan-out with JoinSet, Semaphore, CancellationToken. |
output | JSON serialization and stdout writing (ONLY module with println!). |
platform | Cross-platform initialization (UTF-8 on Windows, TTY detect). |
types | Shared structs and enums. |
error | Error codes and exit codes. |
content | HTTP + readability extraction for --fetch-content (iter. 5). |
fetch_conteudo | Parallel fan-out + per-host rate-limit (iter. 5 / 6). |
selectors | Loading of external ConfiguracaoSeletores (iter. 6). |
signals | Cross-platform signal handlers (SIGPIPE, Ctrl+C). |
config_init | init-config subcommand (iter. 6). |
paths | Path validation and sanitization for I/O. |
browser | Headless Chrome cross-platform under feature chrome (iter.7). |
§Entry Point
The public function run is called by main.rs and returns an exit code
as specified in section 17.7 of the specification.
Modules§
- browser
- Cross-platform detection and launch of headless Chrome via
chromiumoxide. - cli
- CLI argument definitions via
clapderive. - config_
init - Implements the
init-configsubcommand — copies TOMLs embedded in the binary to the user’s configuration directory, allowing local editing without recompiling. - content
- Full text content extraction from URLs (flag
--fetch-content). - error
- Structured error codes as defined in specification section 14.3.
- extraction
- Extraction of search results from DuckDuckGo HTML.
- fetch_
conteudo - Parallel fan-out for content extraction (flag
--fetch-content). - http
reqwest::Clientconstruction and User-Agent selection.- output
- Formatting and emission of the final result to stdout or a file.
- parallel
- Multi-query parallelism with
JoinSet,Semaphore, staggered launch andCancellationToken. - paths
- Path validation and sanitization for I/O operations.
- pipeline
- Orchestration of the CLI execution flow.
- platform
- Platform detection and cross-platform initialization.
- search
- URL construction and search request execution for DuckDuckGo.
- selectors
- Lazy loading of
ConfiguracaoSeletoreswith precedence: - signals
- Cross-platform signal handlers for the CLI binary.
- types
- Shared data types used across the application.
Functions§
- run
- Library entry point. Called by
main.rs.