daedra 0.3.7

Daedra is a self-contained web search MCP server written in Rust. Multiple search backends with automatic fallback. Works from any IP — datacenter, VPS, residential. No API keys required for basic search.

Why Daedra?

Every major search engine (Google, Bing, DuckDuckGo, Brave) blocks datacenter/VPS IPs with CAPTCHAs since 2025. Daedra solves this with a multi-backend fallback chain that automatically finds a backend that works:

Serper (API) → Tavily (API) → Bing RSS → Bing → Google News → Hacker News → Google → Wikipedia → StackOverflow → GitHub → Wiby → DDG Instant → DuckDuckGo HTML

Pure Rust. If one backend is blocked or rate-limited, the next one takes over automatically. Per-backend circuit breakers and governor rate limits keep the chain stable under load; transient failures get a classified retry (exponential backoff, not a blind fixed delay).

Features

• 13 search backends with automatic fallback (see table below)
• Circuit breaker (BackendHealth) — opens after repeated failures, 30s cooldown
• Per-backend keyed rate limiting via governor (API vs knowledge vs scraper tiers)
• Classified retry — only transient errors are retried; bot protection and rate limits fail fast
• Readability extraction — dom_smoothie article body extraction for HTML pages
• PDF support — infer MIME sniffing + pdf-extract text extraction
• Content classification — FetchedContent enum (Html / Pdf / Binary) on fetch
• URL classification — src/url_classification.rs maps search result URLs to content types
• MCP tools — web_search, visit_page, crawl_site (+ search_duckduckgo alias)

Install

cargo install daedra

Search backends

Backends are tried in order. First one that returns results wins.

Usage

MCP Server (for Claude, Cursor, pawan, etc.)

{
  "mcpServers": {
    "daedra": {
      "command": "daedra",
      "args": ["serve", "--transport", "stdio", "--quiet"]
    }
  }
}

CLI

# Search
daedra search "rust async runtime" --num-results 5

# Fetch a webpage as Markdown (HTML via Readability, PDF via pdf-extract)
daedra fetch https://rust-lang.org

# Check backend health
daedra check

# Server info
daedra info

As a Rust library

use daedra::tools::SearchProvider;
use daedra::types::SearchArgs;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let provider = SearchProvider::auto();
    let args = SearchArgs {
        query: "rust programming".to_string(),
        options: None,
    };
    let results = provider.search(&args).await?;
    for r in &results.data {
        println!("{} — {}", r.title, r.url);
    }
    Ok(())
}

MCP Tools

web_search

Search the web with automatic backend fallback.

{
  "query": "search terms",
  "options": {
    "region": "wt-wt",
    "safe_search": "MODERATE",
    "num_results": 10,
    "time_range": "w"
  }
}

Aliases: search_duckduckgo (backward compat)

visit_page

Fetch and extract page content as Markdown. HTML pages use dom_smoothie Readability extraction; PDFs are detected via infer and text is extracted with pdf-extract.

{
  "url": "https://example.com",
  "selector": "article.main",
  "include_images": false
}

crawl_site

Crawl a site from a root URL (sitemap or link following), returning Markdown per page.

Architecture

Daedra
├── SearchProvider (fallback chain, circuit breakers, keyed rate limits)
│   ├── SerperBackend / TavilyBackend (API, optional keys)
│   ├── BingBackend (HTML scraping)
│   ├── WikipediaBackend / StackExchangeBackend / GitHubBackend
│   ├── WibyBackend / DdgInstantBackend
│   └── SearchClient (DuckDuckGo HTML, last resort)
├── FetchClient (FetchedContent: Html / Pdf / Binary → Markdown)
│   ├── dom_smoothie (Readability), infer (MIME), pdf-extract (PDF)
├── url_classification (search result URL → ContentType)
├── SearchCache (moka async cache)
├── MCP Server (DaedraHandler: handle_web_search, handle_visit_page, handle_crawl_site)
│   ├── STDIO transport (JSON-RPC)
│   └── SSE transport (Axum HTTP)
└── CLI (Commands::run, CheckReporter)

Key dependencies

Configuration

# Optional API keys (improves result quality)
export SERPER_API_KEY=...     # Google results via Serper
export TAVILY_API_KEY=...     # AI-optimized search
export GITHUB_TOKEN=...       # Higher GitHub API rate limit

# Logging
export RUST_LOG=daedra=info

Ecosystem

Built by DIRMACS.

License

MIT