Skip to main content

Crate lockdocs_core

Crate lockdocs_core 

Source
Expand description

lockdocs core: read lockfiles, find each dependency’s installed sources, extract API reference and prose, and answer queries with BM25 under a token budget. Public release-tag docs enrich first-use queries by default; opt-out and offline controls keep installed/cached answers local.

Re-exports§

pub use semantic as embed;

Modules§

bm25
BM25 over entries (adapted from repomap’s engine), with a light English stemmer and query stopwords so questions in plain language rank well.
cache
On-disk cache location and a stable hash for cache keys.
extract
Turn a package’s files into entries: API symbols (signature + doc comment) from TypeScript declarations, JavaScript, Python, Rust and Go sources, and prose sections from READMEs, changelogs and docs folders.
fetch
Opt-in network fetch of one exact package version from its registry (npm tarball, PyPI wheel or sdist, crates.io .crate, Go module proxy zip), unpacked into the disk cache. Nothing here runs unless fetching is enabled.
index
Per-package index: entries plus their term frequencies, cached on disk by package, exact version and source location.
locate
Find a dependency’s installed sources on this machine: node_modules, Python site-packages, the Cargo registry and the Go module cache.
lockfile
Lockfile and manifest parsers. Each parser is a pure function from file contents to exact versions, so they are easy to test.
markdown
Split Markdown, reStructuredText and plain text into heading-scoped sections small enough to rank and quote.
project
A project: its lockfiles and the exact dependency versions they pin.
query
The three questions agents ask: which versions do I have (resolve), what do the docs say about X (docs), and what exactly is this symbol (api). Answers are token-budgeted and cite package@version path:line.
semantic
Product policy for the shared embedding engine: the retrieval model, identifier-aware inputs, lockdocs’ opt-out and existing cache location.
tokenize
Code-aware tokenizer: identifiers are kept whole and also split on camelCase, PascalCase, snake_case and digits, all lowercased.
upgrade
Upgrade report (lockdocs Pro): the API difference between the pinned version of a package and a target version, limited to the symbols the project actually uses, with call sites and migration-guide pointers.
upstream
Upstream docs: many packages ship no documentation (Next.js, Django, FastAPI…). Their repositories do, under the exact version’s git tag. This module finds the repository from the package’s own metadata, finds the tag for the pinned version, and downloads only the docs folders (Markdown, MDX, reStructuredText) from GitHub into the cache, once. First-use queries fetch public docs at an immutable release commit. Explicit fetch also supports major-version docs sites and optional GitHub credentials.

Structs§

Dep
A resolved dependency: an exact version from a lockfile or an install.

Enums§

Eco
Package ecosystems lockdocs understands.

Functions§

est_tokens
Conservative token estimate used for budgets: 3 characters per token, which keeps answers within budget under o200k/cl100k tokenizers for code-heavy text (the benchmark measures with a real tokenizer).
norm_name
Normalize a package name for comparison within an ecosystem.