domain-status
domain-status is a concurrent URL/domain scanner. Give it a list of URLs; it captures HTTP status, TLS certificates, DNS, technology fingerprints, and related signals in one pass, and stores results in SQLite for relational analysis (1:many satellites for technologies, secrets, redirects, DNS rows, and more). Export the same run to Parquet or JSONL when you want DuckDB / data-lake workflows or shell pipelines—SQLite remains the source of truth.
Who it's for: DevOps/SRE, security analysts, threat-intel / OSINT researchers, and anyone managing large URL/domain lists who wants one tool instead of stitching curl, whois, and Wappalyzer together.
Scan only hosts you are authorized to scan.
Quick Start
# Install (macOS/Linux)
&&
# or: cargo install domain-status
# or: download a binary from Releases
# Convenience digest (same DB)
# SQLite is the primary result store (1:many joins are intentional)
More SQL: QUERIES.md and DATABASE.md.
Optional exports (from the same SQLite DB—lossy flatten vs full schema for CSV/JSONL):
# Shell tooling (payload on stdout; logs/banners on stderr — see docs/CLI.md)
|
# Columnar analytics (e.g. DuckDB: SELECT * FROM 'recon.parquet')
CSV: domain-status export --format csv. Everyday flags: domain-status scan -h. All flags: --help.
WHOIS is on by default (--no-whois to disable). GeoIP needs MAXMIND_LICENSE_KEY or --geoip. First-party script bodies: --scan-external-scripts. Local fingerprint rules, caches, and GitHub-free ruleset loading: docs/ADVANCED.md.
Features (core)
- HTTP status, redirects, response metadata
- TLS certificate fields (resolves all public IPs, IPv4-first, so dual-stack hosts with broken IPv6 egress still get a certificate)
- DNS (NS/TXT/MX + SPF/DMARC/MTA-STS/TLS-RPT/BIMI), parsed HSTS, CDN taxonomy,
security.txt/robots.txt - Technology fingerprints + exposed secrets (static analysis; no JS execution). Fingerprints use headers/cookies/meta/HTML/
scriptSrcURLs/inline script text/DNS/cert issuer from the initial response. With--scan-external-scripts, first-party fetched script bodies also feed secret detection and Wappalyzerscriptstech patterns. - Security signals are query-derived (no
url_security_warningstable): e.g. non-HTTPS finals, weak TLS versions, self-signed/mismatched/expired certs. Header presence (HSTS/CSP/etc.) is stored; "missing header" is a query, not a precomputed warning — see DATABASE.md and QUERIES.md §30 - SQLite fact table + satellite tables;
summarycommand - Concurrency / rate limiting;
--fail-onfor CI exit policy
Also available (see docs): GeoIP, WHOIS, live status/Prometheus server, CSV/JSONL/Parquet export, TOML/DOMAIN_STATUS_* config.
Limitations
- Static analysis only — no JavaScript rendering; JS-runtime-only fingerprint rules will not match
- Fingerprint rules depend on GitHub refresh, cache, or a local
--fingerprintspath (target scans still need network) - Private/link-local targets are blocked (SSRF protection)
Configuration (keep it simple)
Day-to-day: CLI flags (domain-status scan -h; all flags: scan --help). Prefer -v / -q for log verbosity.
Repeatable jobs: pick one of TOML (--config ./domain_status.toml or cwd domain_status.toml) or DOMAIN_STATUS_* env for the job profile; use CLI only to override. Mixing TOML and env for the same knobs works (env wins) but is harder to reason about. --config wins over DOMAIN_STATUS_CONFIG_FILE.
Local secrets: put MAXMIND_LICENSE_KEY / GITHUB_TOKEN in .env or the real environment — never in TOML. .env only seeds process env; it is not a fifth precedence tier.
Precedence: CLI > DOMAIN_STATUS_* > TOML > defaults. Details: docs/CLI.md.
Example TOML: config_examples/domain_status.example.toml.
Documentation
| Doc | Purpose |
|---|---|
| docs/CLI.md | Command cheat sheet (scan / summary / export) |
| docs/ADVANCED.md | GeoIP, WHOIS, fingerprints, status server, caches, tuning |
| DATABASE.md | Full schema |
| QUERIES.md | SQL cookbook |
| docs/EXIT_CODES.md | CI exit policies |
| docs.rs/domain-status | Library / embed API |
| CONTRIBUTING.md / docs/DEVELOPER_BOOTSTRAP.md | Development |
License
MIT for this codebase — see LICENSE. For Rust dependencies and optional runtime data (fingerprints, gitleaks rules, MaxMind), see docs/LICENSES.md.