paper-gap
paper-gap is a local, on-demand Rust CLI for finding research papers that appear to lack useful public code, or only have weak/prototype/stale implementations.
It is not a recommender. It does not rank by user preference, hardware fit, or subjective interest.
What it does
- Fetches papers from arXiv and OpenAlex.
- Normalizes paper metadata.
- Checks Papers with Code for known code links.
- Searches public code forges:
- GitHub
- GitLab
- Forgejo/Gitea, with Codeberg enabled by default
- Produces markdown or JSON reports.
- Explains the transparent additive implementation-gap score.
- Ranks repository matches with deterministic confidence and matched evidence.
- Lists configured and discoverable sources.
Install / run
Build and install paper-gap so it is on your PATH:
Then run:
Examples
Scan recent or queried papers:
OpenAlex handles date-only scans and participates in query scans. Use --paper-source all|arxiv|openalex to select paper providers; all is the default. Categories are arXiv-only, while date-only scans are OpenAlex-only. Use --repo-source all|github|gitlab|codeberg|none to control heuristic repository search; all is the default. none keeps Papers with Code links and supported official repository evidence without treating disabled search as proof of absence.
Inspect one arXiv paper:
Write JSON, then render a markdown report later:
List sources:
Discover possible new sources, informational only. Set BRAVE_SEARCH_API_KEY to use the official Brave Search API; otherwise discovery falls back to DuckDuckGo HTML.
Output
Reports are written to stdout unless --output is supplied. Progress messages go to stderr so stdout can be piped safely.
Markdown reports are ranked by implementation_gap_score. JSON reports include schema_version, scan or inspect provenance, underlying papers, code links, repository results, score components, and gap explanations. paper-gap report continues to accept pre-0.3 JSON without provenance.
Scoring
The score is additive and intentionally simple:
implementation_gap_score =
no_code_score
+ prototype_only_score
+ stale_repo_score
+ packaging_gap_score
+ docs_gap_score
+ tests_gap_score
+ license_gap_score
+ reproducibility_gap_score
+ ecosystem_gap_score
Each non-zero score should have an explanation in the report.
Repository match confidence
Repository match confidence is deterministic evidence, not a probability. Exact official links, arXiv IDs, and DOIs score 100; exact titles score 60; partial title, description, and specific method terms add smaller amounts. Matches below 30 are rejected. Generic acronyms and substring-only collisions do not count as evidence. The behavior is checked against an offline positive/negative fixture corpus.
API notes and rate limits
paper-gap is local and only runs when you invoke it, but it still calls public APIs. Provider requests retry once after connection failures, timeouts, HTTP 502/503/504, or HTTP 429. Integer Retry-After values are honored up to five seconds; permanent client errors are not retried. If arXiv continues to rate-limit a scan:
- wait a few minutes;
- retry with a smaller
--limit; - prefer targeted
--category,--query, or date ranges.
For broader scholarly metadata, query scans also use OpenAlex. OPENALEX_EMAIL is optional and identifies requests to the OpenAlex polite pool.
Repository searches can also hit anonymous rate limits. Optional tokens are supported via environment variables:
Current MVP limits
- Background jobs, local database storage, and a web dashboard are optional future features, not required defaults.
- LLM judgement, subjective preference ranking, and hardware-fit ranking are optional future experiments, not part of the default score.
- Source discovery optionally uses Brave Search API and otherwise falls back to DuckDuckGo HTML; it remains informational only.
- Repository search is heuristic; reports expose deterministic match confidence and evidence rather than claiming certainty.