docsrs-cli
Fetch crates.io and docs.rs docs in one shot for agents.
What is it
- One-shot stdin/stdout CLI for crates.io search and docs.rs item pages
- Lifecycle is always BORN, EXECUTE, FINALIZE, DIE
- No daemon, no sticky session, no product telemetry
- JSON auto-selects when stdout is not a TTY
- Public HTTP only against an allowlist of docs hosts
- Current product line is 1.3.x (release 1.3.0)
The Pain
- Agents scrape HTML by hand and burn tokens on noise
- Browser tabs and curl pipelines do not form a stable contract
- Sticky MCP servers keep sockets open between turns
Why
- Stable JSON envelopes on stdout for every command
- Markdown human path when you force
--format markdown - XDG disk cache with TTL and soft budget plus
data.cache_hit - Polite rate limits, full-jitter HTTP retry, cancel-safe shutdown
- Exit codes agents can branch on without parsing prose
- Ranked in-crate match modes stop noisy false positives
Superpowers
search-cratesagainst crates.io with pagination, sort, and--page-token--page-tokenechoesquery/page/per_page/sortfrom the effective URLreadmefor the docs.rs crate overview docblock withresolved_versionget-itemfor typed rustdoc pages including associated methods- Method fetch sets
data.extractiontomethodon success; missing#method.Xisnot_found(exit 66), never a false parent-page success --suggeston method 404 ranks nearby method leaves from the parent type page (and all.html for other kinds)- Budgets above hard max fail closed with exit 65 (no silent clamp)
- Error envelopes include
commandandduration_mslike success envelopes search-in-crateoverall.htmlwith--match exact|prefix|substring--suggestranks exact → prefix → substring → edit-distance on get-item 404, returned aserror.suggestions[{path,kind}]item_pathaccepts hyphens and normalizes to underscores for rustc paths- Markdown scrub removes rustdoc chrome (
§, “Copy item path”) - Body over cap is
error.kind=budget(exit 74,retryable=false) - stdlib fetch for
std,core, andallocvia doc.rust-lang.org commands,schema,doctor,versionfor agent discoverydoctor --onlinefor opt-in DNS probes to crates.io and docs.rsdoctortop-levelokmirrorsdata.okcacheandconfigfor XDG maintenance without secrets
What's New in 1.3.0
--sort-byand--max-itemscomplete the reduction pipeline: filter → sort-by → dedupe-by → max-items → select → count-only → truncate-content → max-output-bytesagent_surfacegainedlimited, so a small result is distinguishable from a capped oneagent-surfaceschema is published;schema --cmd allcarries 20 wire namesget-itemreaches enum variants and struct fields (variant,structfield), plus trait associated items and required trait methodserror.suggestionspublishes the--suggestranking as data, so no agent parseserror.messageproseanchor_familynames the real rustdoc anchor family behind amethodextractionlog_directiveconfig key steers stderr verbosity; an unparseable value is rejected at load (exit 78)RUST_LOGis no longer read: it used to outrank the CLI, which is a product knob living in env- TLS trust anchors are back to bundled
webpki-rootsafter areqwestupgrade had silently moved them to the OS store - Policy gates are Rust integration tests (
cargo test --test policy_gates), so they run on Linux, macOS and Windows alike - Full notes under CHANGELOG.md section
[1.3.0]
Earlier highlights (1.2.0)
- Method fail-closed: missing
#method.Xreturnsnot_found(exit 66), not a parent-page false success --suggestranks method leaves from the parent type HTML on method typos- Error envelopes carry
command+duration_ms(parity with success) - Values above hard max for body/output budgets fail with exit 65 (no silent clamp)
- Method 404
source_urlkeeps the first probe kind (struct), not the last - Dry-run documents
validation=url_shape_onlyand parent kind probes - Offline schema files match
schema --cmd all(19 wire names including aliases at that release) - Full notes under CHANGELOG.md section
[1.2.0]
Earlier highlights (1.1.x)
- Short method paths (
Runtime::new) resolve via all.html parent type lookup crate@versionsugar on readme / get-item / search-in-crate- Effective-URL echo for
--page-token, cascade--suggest, hyphen normalize - Fail-closed
--timeout 0/--connect-timeout 0(exit 65) - Non-retryable local body budget errors (
kind=budget, exit 74) - Product knobs: CLI flags + XDG only (no product
DOCSRS_CLI_*env)
Quick Start
Installation
- From crates.io:
cargo install docsrs-cli --locked - From a local checkout:
cargo install --path . --locked - MSRV is Rust 1.88
- No Cargo feature flags exist in this package
TLS
- rustls only (no
native-tls/ OpenSSL runtime); crypto providerring - Minimum protocol TLS 1.2; peers may negotiate TLS 1.3
- Trust store: webpki-roots (Mozilla); certificate validation always on
- Production hosts require HTTPS (allowlist); offline tests may use loopback HTTP
- No
danger_accept_invalid_*, no product KeyLog, no mTLS (public docs origins) - Runtime posture:
docsrs-cli doctor --json→ checkhttp_client_posture - Decision record:
docs/decisions/0007-rustls-posture.md
Usage
- Pass
--jsonor pipe stdout for the agent envelope - Force human output with
--format markdownor--format text - Plan URLs without network using
--dry-run - Raise wall-clock budget with
--timeout <seconds> - Disable disk cache with
--no-cache - Product knobs use CLI flags or XDG
config.toml, never product env vars
Commands
- Full surface has 11 top-level commands
search-crates [query] [--page N] [--per-page N] [--sort KIND] [--page-token TOKEN]- Query may be omitted when
--page-tokencarries a full query string --pageconflicts with--page-token--sortacceptsrelevance,downloads,recent-downloads,recent-updates,new,alphabeticalreadme <crate> [--crate-version V]— docs.rs crate overview docblockget-item <crate> <item_type> <item_path> [--crate-version V] [--suggest]item_typeacceptsmodule/mod,struct,trait,enum,union,fn/function/method,type,const/constant,static,macro,attr/attribute,derive,variant,structfield/fieldvariantandstructfieldhave no page of their own: qualify asParent::leafor exit 65 names the parent kindsmethodis an alias offnfor associated methods- Associated methods resolve to parent type pages with
#method.nameanditem_name - Successful method fetches set
data.extractiontomethod; missing anchors arenot_found(exit 66) item_pathaccepts::or/separators and optional crate prefix--suggeston get-item 404 lists nearby symbols (method leaves from parent page; other kinds fromall.html)search-in-crate <crate> [query] [--crate-version V] [--item-type K] [--limit N] [--match MODE]--matchacceptsexact,prefix(default),substring- Empty
querylists classified items up to--limit(clamped to 1000) - Hits may include
scorewhen a query is present version— binary versiondoctor— local TLS, paths, concurrency, and retry readinessdoctor --online— also probes crates.io and docs.rs DNScommands— full command tree for agentsschema --cmd <name>— JSON Schema for a command payload- Schema targets:
search-crates,readme,get-item,search-in-crate,version,doctor,commands,schema,completions,error,dry-run,agent-surface,cache,configplus aliases (cache-path,cache-clear,cache-stats,config-path,config-show,config-init); useschema --cmd all --jsonfor the full bundle completions <shell>— raw shell script by default; JSON only with explicit--json- Shells:
bash,zsh,fish,elvish,power-shell(powershellalias) cache path— print the resolved cache root, its winning layer (cli/xdg/unresolved) andno_cachecache stats— report entry count, bytes, and budgetcache clear— delete cached HTTP bodiesconfig path— print resolved config/cache dirs and winning layerconfig show— print effective runtime configurationconfig init [--force]— create defaultconfig.toml
JSON contract highlights
- Success envelope:
schema_version,ok,command,data,duration_ms - Network payloads use canonical
crate_name(nevercrate) - Network payloads expose
cache_hitfor local disk cache only get-itemexposesitem_name, optionalresolved_version; method success includesextraction=method- Agents MUST reject method success when
extractionis missing or is the legacyitem_pagevalue anchor_familycarries the real rustdoc family, becauseextractionreportsmethodfor variants and struct fields tooreadmeexposes optionalresolved_version(stdlib channel isstable)search-in-crateechoesmatch_modeand optionalitem_typesearch-cratespagination tokens live underdata.meta.next_page/prev_page- After
--page-token, echoedquery/page/per_page/sortmatch the effective URL - Failure envelopes expose
schema_version,ok:false,command,duration_ms, and nestederror(kind,retryable, …) - Never retry
kind=budget(exit 74); raise--max-body-bytesonly within the hard max (above hard max is exit 65)
Environment Variables
- Paths: use
--config-dir/--cache-dir(or platform XDG viadirectories) - Locale: use
--langor TOMLlang(never product env) - Product knobs (timeout, UA, cache TTL, retries, concurrency, lang, paths) are never read from
DOCSRS_CLI_*env at runtime - Use CLI flags and XDG
config.tomlfor product settings RUST_LOGis not read: stderr verbosity comes from-q/-vor the TOML keylog_directive- Terminal capability only:
NO_COLOR,TERM,CLICOLOR_FORCE— they describe the device, likeisatty, and carry no product configuration - Transport only:
HTTP(S)_PROXY/NO_PROXY, honored byreqwestitself, never by a product knob
Payload Reduction
- The CLI cuts the payload before serialization; no
jq/jaqstage is needed - Project keys:
docsrs-cli --select planned_url --dry-run readme serde --json(alias--fields) - Missing keys are skipped, never emitted as null
- Filter:
key=value,key!=value,key~substring; repeat--filterfor AND - A malformed
--filterexits65; a typo never looks like an empty result - Sort with
--sort-by <KEY>(stable, ascending; elements without the key go last) - Numbers compare numerically, so
9sorts before10, never after it - Limit emission with
--max-items <N>; it bounds the output, not the query search-in-crate --limitbounds the query instead: it decides how much gets classified- Both exist because they are different bounds, and clap rejects two flags sharing a name
- Deduplicate with
--dedupe-by <KEY>; count with--count-only - Shorten strings with
--truncate-content <N>(characters, never split UTF-8) - Cap the whole envelope with
--max-output-bytes <N>: it drops whole hits, never bytes, so the JSON stays parseable - Measured on
search-in-crate serde "" --limit 200:--max-output-bytes 2000emits 1973 bytes and 12 of 62 hits - Used alone it does not activate the pipeline, so
agent_surfaceis absent anddata.truncatedcarries the signal - Order: filter, sort-by, dedupe-by, max-items, select, count-only, truncate-content, max-output-bytes
- Sorting before dedupe decides which duplicate survives; limiting after it protects the slots
--count-onlytherefore counts what survived the filter, the dedupe and the limitagent_surfacereportsinput_count,output_count,limited,content_truncated,output_truncatedlimitedseparates a genuinely small result from one that--max-itemscut- Sibling counters that name the array follow it:
emittedis rewritten,totalnever is - Top-N without a pipe:
docsrs-cli --filter kind=struct --sort-by name --max-items 5 --select name search-in-crate serde "" --limit 200 --json - Full contract:
docsrs-cli schema --cmd agent-surface --json
Integration Patterns
- Agent subprocess:
docsrs-cli get-item serde trait Serialize --json - Method fetch:
docsrs-cli get-item tokio fn runtime::Runtime::new --json - Ranked symbols:
docsrs-cli search-in-crate serde Serialize --match prefix --json - Paginate: read
meta.next_pagethendocsrs-cli search-crates --page-token '...' --json - Discovery first:
docsrs-cli commands --jsonthenschema --cmd get-item --json - Offline plan:
docsrs-cli --dry-run readme tokio --json - See INTEGRATIONS.md and docs/AGENTS.md
Performance
- One primary GET per command in the happy path
- Multi-thread Tokio runtime with Semaphore concurrency budget
spawn_blockingfor large HTML parse work- Disk cache avoids repeat downloads within TTL
- Warm cache responses set
data.cache_hitto true
Memory Requirements
- Default body cap is 10 MiB per response
- Default output cap is 2 MiB per emission
- Default disk cache budget is 256 MiB soft
- Raise caps only with explicit CLI flags or XDG
config.toml; values above hard ceilings fail closed (exit 65), never silent clamp
Troubleshooting FAQ
- Exit
65means invalid input (including explicit--timeout 0/--connect-timeout 0, or budget flags above hard max) - Exit
66means the crate or item was not found (including missing method anchors) - Use
get-item ... --suggestto list nearby symbols after a 404 (method typos include parent method leaves) - Exit
69means rate limit or temporary upstream outage (retryable) - Exit
74witherror.kind=networkmeans transport failure; retry with backoff - Exit
74witherror.kind=budgetmeans local body cap; do not retry — raise--max-body-bytes - Always read
error.retryablebefore retrying any non-zero exit - Exit
78means local config or path readiness failed - Exit
124means wall-clock timeout - Run
docsrs-cli doctor --jsonbefore blaming the network - Treat doctor healthy only when top-level
okanddata.okare both true - Run
docsrs-cli doctor --online --jsonto probe crates.io and docs.rs DNS - Noisy search hits: switch from
--match substringtoprefixorexact - Hyphenated paths: pass
async-traitsegments; the CLI normalizes toasync_trait
Documentation Map
- How to use
- Agents
- Cookbook
- Configuration — every flag and every
config.tomlkey - Cross platform
- Migration
- Testing
- JSON schemas
- Architecture decisions — nine ADRs, each with a pt-BR pair
- Integrations
- Security policy
- Contributing
- Changelog
- llms.txt
Contributing
- Read CONTRIBUTING.md
- Follow the Code of Conduct
Security
- Read SECURITY.md
- Report issues privately to daniloaguiarbr@proton.me
Changelog
- See CHANGELOG.md for version history
- Current release notes for 1.3.0 live under
[1.3.0]
License
- Dual-licensed under MIT or Apache-2.0
- See LICENSE, LICENSE-MIT, and LICENSE-APACHE