# rd-rds
`rd-rds` is a scoped, read-only reader for installed-R-package information and
selected CRAN-like repository indexes. It is not a general R serialization
library; strict decoding rejects unknown SEXPs. Its compression support is
implemented in pure Rust, so the crate links no C library. See the [workspace
README](../../README.md) for repository status and crate relationships.
The API provides these entry points:
- `parse` reads a decompressed XDR serialization stream only.
- `inspection::inspect` observes a decompressed stream's root kind and closure
formals without constructing an object or validating its complete payload.
- `file::from_bytes` and `file::read` accept the complete envelope and apply
bounded decompression. Supported envelopes are raw `X\n` XDR, gzip, xz,
bzip2, and zstd (when the corresponding feature is enabled).
- `package` provides validated convenience views for `Meta/package.rds` and
CRAN-like `PACKAGES.rds` matrices.
- `matrix::CharacterMatrix` provides a validated, owned view of general R
character matrices, including matrices without `dimnames`.
With the opt-in `lazyload` feature, [`lazyload::LazyLoadIndex`] reads an `.rdx`
index without requiring its companion `.rdb` data file. Its `open` and
`open_with_options` constructors expose stored variables, persistence-reference
descriptors, and record compression. Variables and references retain their
index order and duplicate names; name lookups return the last matching entry.
For example, callers can enumerate code or lazy-data names with
`LazyLoadIndex::open(path)?.variables()` without loading any records.
`Options::max_index_bytes` bounds both stored and decompressed index bytes,
each to 256 MiB by default. The index reader also applies the default RDS
decoder limits. Record-size options apply only to record reads. Index parsing
validates descriptors; it does not check their ranges against a data file.
The returned `Compression` describes records, independently of the index
file's compression envelope.
[`lazyload::LazyLoadDb`] uses the same index reader and opens the `.rdx`/`.rdb`
pair to read direct records without an R session. The record layer recognizes
uncompressed and zlib records. bzip2 and xz records are recognized in the
index, but reading them reports an explicit unsupported error. Compound
persistence references are described by their eager and lazy record locations
but are not resolved. Unrecognized persistence descriptors are retained as
`RecordReference::Unsupported`. Record reads return both the exact addressed
bytes and the decompressed payload, with independent 256 MiB default bounds.
Raw records are already XDR bytes and do not carry a length prefix; zlib
records carry a four-byte declared length.
With the same feature, [`package::InstalledCodeDb`] provides the package-level
API for that pair. The caller supplies the installed package directory; the
reader selects `R/<basename>.rdx` and `.rdb` and does not discover libraries or
scan runtime exports. [`InstalledCodeDb::stored_bindings`] is the complete
`variables` map in index order, including duplicate names. Consequently this
view's completeness domain is `CodeDatabaseVariables`, not exports or runtime
bindings. [`InstalledCodeDb::inspect_stored_binding`] returns structured
unknown/ambiguous errors instead of choosing among duplicate names, reads only
a unique direct record, and returns bounded closure prefix metadata. Closure
bodies are reported as `BodyValidation::NotValidated`.
The selected record is fully loaded, decompressed, and checked by the
`LazyLoadDb` container layer first; stored/decompressed size limits, framing,
compression, and trailing-stream corruption therefore fail before prefix
inspection. `InstalledCodeOptions::max_bytes_visited` applies only to the
semantic prefix walker, which stops before reading or validating a closure's
decompressed body payload.
The provenance includes both selected paths, compression, and an opaque
`CodeDbGeneration` derived from best-effort file metadata. It is an identity
hint rather than a content hash or a transaction guarantee.
The `lazyload` feature includes the `gzip` feature because installed package
`.rdx` indexes use the standalone gzip envelope in the normal package
profile. Callers that enable `lazyload` therefore also get gzip `.rds`
handling; the other standalone codecs remain independently selectable.
The low-level `lazyload::decode_stored_record` helper applies the same bounded
record decoder to an already isolated `(offset, length)` byte slice.
Record reads take a metadata snapshot when the database opens and compare it
before and after each read. On Unix this includes device and inode, and on
other platforms it uses file length and modification time when available.
This is best-effort detection of replacement or concurrent modification and
cannot provide a transaction guarantee against races after the final check.
Neither the generation nor replacement detection is a content hash or a
portable guarantee: a replacement that preserves all observed metadata may
not be distinguishable on every platform.
The [`rd-helpdb`](../rd-helpdb/README.md) crate uses the file layer for
standalone help-database RDS files, and [`rd-ast`](../rd-ast/README.md) can
lower supported decoded documentation objects into the common document model.
## Namespace metadata
[`package::NamespaceMetadata`] provides an owned view of declarations from a
decoded `Meta/nsInfo.rds` object. Its exports, imports, S3 registrations, and
S4 declarations are static metadata: they are not runtime namespace exports,
stored lazy-load bindings, evaluated export patterns, or `.onLoad` results.
Each known field is independently represented by [`package::MetadataField`],
so a malformed S3 schema does not hide valid declared exports.
Declared exports are returned as [`package::NamespaceExport`] values: the
source binding and namespace-facing name are preserved separately, so an
assignment-shaped export such as `export(public = internal)` is not confused
with an ordinary `export(name)`. Empty export-name attributes use the source
name. The installed-package fixture also exercises the R-written
`list("utils", except = ...)` import shape and an aliased `importFrom`.
A consumer such as a mini-roxygen provider can retain its existing policy
boundary while replacing ad-hoc S3 extraction with positive evidence:
```rust
use std::collections::BTreeSet;
use rd_rds::package::{MetadataField, NamespaceMetadata};
fn generic_evidence(object: &rd_rds::RObject) -> BTreeSet<String> {
match NamespaceMetadata::from_object(object)
.ok()
.map(|metadata| metadata.s3_generic_evidence().clone())
{
Some(MetadataField::Present(generics)) => generics.into_iter().collect(),
Some(MetadataField::Missing)
| Some(MetadataField::Invalid(_))
| Some(MetadataField::UnsupportedSchema { .. })
| None => BTreeSet::new(),
_ => BTreeSet::new(),
}
}
```
The caller still decides how missing metadata, invalid schemas, base-generic
catalogs, shadowing, and library precedence should affect its provider.
### Installed metadata paths
Consumers that already have an installed package directory can use
[`package::NamespaceMetadata::read_installed`] and
[`package::PackageMeta::read_installed`] to read the canonical
`Meta/nsInfo.rds` and `Meta/package.rds` artifacts. These helpers only join the
canonical path, perform the bounded file read, and construct the typed view;
they do not discover packages, resolve exports, or model runtime state.
`read_installed_with_options` is available on both views when a consumer needs
explicit [`file::ReadOptions`] bounds. [`package::InstalledMetadataError`]
keeps file-layer failures distinct from typed-view validation failures and
retains the selected artifact path in either case.
`PackageMeta::built().r_version()` and
`PackageMeta::description_field("Priority")` expose the validated `Built.R`
and `DESCRIPTION` metadata used by consumers such as base-package catalogs.
An absent DESCRIPTION field is not the same as a present `NA` field, while an
absent `Built` element is represented by `PackageMeta::built() == None`.
## Closure prefix inspection
`inspection::inspect` and `inspection::inspect_with_options` expose the bounded
inspector without requiring optional features. They accept decompressed XDR
bytes starting with the serialization header. A consumer
that already loaded a lazy-load record can inspect and decode the same bytes
without reopening the database or decompressing twice:
```rust,ignore
let record = database.read(binding_name)?;
let observed = rd_rds::inspection::inspect(record.decompressed_bytes())?;
// A consumer can retain its decoder for expressions the inspector omits.
let object = consumer_decode(record.decompressed_bytes())?;
```
The caller retains path selection and duplicate-name policy. Inspection does
not look up stored bindings, evaluate promises, or resolve persistent references.
`InstalledCodeDb::inspect_stored_binding` delegates to the same inspector after
its own lookup and container checks. Existing inspection types at `package::*`
remain available with `lazyload` and are the same types as `inspection::*`.
For a closure, inspection walks attributes, environment fields, and the
formal/default pairlist while
sharing strict decoding's reference registration/resolution, encoding, depth,
and element accounting. It also applies inspection-specific byte and
formal-count limits through `InspectionOptions`: 256 MiB and one million formals
by default. These bounds do not limit the caller's file reads or decompression.
It reports formal names in wire order (including `...`, duplicate names, and
non-syntactic UTF-8 names), distinguishes missing defaults from present
defaults including `NULL`, and stops immediately after observing the body tag.
The body payload is intentionally not validated. Prefix failures retain their
phase and byte offset in an unavailable result after the root kind is known;
failures before the root flags remain top-level errors. Non-closures return
`InspectionExtent::RootTagOnly`, including unknown root kinds and records whose
payloads are missing or invalid. The root kind describes the stored wire tag;
it does not necessarily describe a materialized value, establish runtime
callability, or prove that the object can be decoded. In particular, ALTREP
values still require decoding to determine the materialized value's kind, and
the shared decoder currently rejects those values as unsupported.
Unvisited body payloads and trailing bytes remain unchecked. Use `parse` when
full decoding is required for a supported object. Deterministic plain and
compiler-produced format-2/format-3 fixtures, along with generated diagnostic,
ALTREP, S4, namespace, and persisted-reference cases, are produced in the source
repository by
`tests/fixtures/generate_closure_inspection_fixture.R`.
## Runnable examples
```text
cargo run -p rd-rds --example inspect_packages -- /path/to/PACKAGES.rds
cargo run -p rd-rds --example inspect_rds -- /path/to/archive.rds
cargo run -p rd-rds --features lazyload --example inspect_installed_package -- /path/to/installed/package [binding]
```
`inspect_packages` demonstrates the typed, stable package-index view.
`inspect_rds` provides a bounded advanced inspection of unfamiliar decoded
objects, including shapes that are not package matrices.
`inspect_installed_package` reads `Meta/nsInfo.rds` with `file::read`, reports
declared S3 generic evidence, and lists or inspects stored code-database
bindings from the explicit package directory. Its output keeps three domains
separate: declared namespace metadata, code-database variables, and runtime
namespace state. It never starts or inspects an R runtime. Consumers should
derive a closure-formals signature only from `FormalsInspection::Available`;
`NotApplicable` (including callable built-ins and specials) and `Unavailable`
remain distinct states.
A missing index is reported as `NoCodeDatabase`, while an existing index with
no variables is a valid empty database. An unknown binding is reported as
`UnknownStoredBinding`, and a duplicate name as `AmbiguousStoredBinding`;
consumers should keep these database states distinct rather than treating them
as an empty or last-wins lookup.
The stored-code view is not a runtime namespace: a stored binding is evidence
about the installed code database only. `DefaultPresence::Absent` means that
the serialized closure has no default expression; it does not by itself prove
that the argument is required (an R function may use `missing()`). Available
formals likewise do not imply that the closure body was validated;
`BodyValidation::NotValidated` remains explicit.
## Repository-index interoperability
The supported contract is the tested decoding behaviour described in the
[workspace stability policy](https://github.com/eitsupi/r-documentation-rs/blob/main/STABILITY.md),
not the continued availability or unchanged schema of files hosted by third
parties. Deterministic fixtures cover these CRAN profiles:
- `src/contrib/PACKAGES.rds`: xz envelope, serialization format 2, and the
17-column main-index schema.
- `src/contrib/Archive/<package>/PACKAGES.rds`: gzip envelope, serialization
format 3, and the 15-column package-archive schema.
- `src/contrib/Meta/archive.rds`: gzip envelope, serialization format 3, and
a named list of `file.info()`-shaped data frames.
Real CRAN examples were compared cell-for-cell with R 4.6.1 `readRDS()` on
2026-08-04; decoded cell values matched in all three profiles. R-universe was
manually verified on 2026-08-05: source indexes used gzip, the Windows and
macOS binary-repository indexes used zstd, and the observed schema had 15
columns with `SHA256` in place of CRAN's `MD5sum`. These observations fall
within the reader's general matrix, encoding, and compression behaviour, but
the test suite contains no R-universe-specific fixture.
These statements describe observed interoperability at the stated dates. They
do not guarantee that an external service retains the same paths, schemas,
compression, or serialization behaviour.
### Upstream archive semantics
These are upstream semantics, not reader guarantees. As observed on
2026-08-04, CRAN's per-package `Archive/<package>/PACKAGES.rds` excludes the
current package version, and its rows are in archival rather than
semantic-version order. Consumers must not infer inclusion of the current
release or version precedence from row position. This is a recently
introduced and undocumented CRAN facility and may change or disappear
independently of `rd-rds`.
### String encoding metadata
`RStr::encoding()` reports the CHARSXP encoding flag stored in the serialized
data. R-universe files are generated by a JavaScript serializer rather than by
R, and currently flag every string as UTF-8, including ASCII-only strings,
while R's `Encoding()` reports those strings as `"unknown"` after `readRDS()`.
Encoding labels can therefore differ even when decoded string contents are
identical; this is not a decoding incompatibility.
R serialization format 2 has no native-encoding field in its header. A
non-ASCII CHARSXP marked Native is therefore ambiguous. The reader preserves
the bytes lazily for retained `RStr` values rather than guessing a locale: with
the default policy, conversion by `RStr::as_str()` or a typed view rejects the
value. A `SYMSXP` print name is converted during parsing instead, so a symbol
name that cannot be decoded fails with `Error::InvalidSymbolName` at parse time
under either policy. A caller with an independent UTF-8 contract may opt in with
`ReadOptions::native_encoding_policy(NativeEncodingPolicy::AssumeUtf8)` (or the
corresponding `parse_with_options` API with `ParseOptions`). The opt-in still
validates retained `RStr` bytes with `str::from_utf8` when conversion occurs and
never performs lossy replacement. The policy applies only when the header
field is absent, which means format 2; a format-3 header value is always
authoritative. `RStrValue::native_encoding_source()` distinguishes a
header-declared encoding from a caller assumption, while
`RStrValue::header_native_encoding()` reports header evidence only.
Unknown or unsupported SEXP values reachable from the decoded result are
hard decode errors; they are never silently converted to a known value. The
one exception is environment internals: environments are collapsed to opaque
handles, and a limited set of verified value shapes inside them (for example
complex, raw, and S4 objects) is wire-consumed and discarded rather than
rejected. Decoder defaults are a depth limit of
5,000, a vector limit of 8,000,000 elements, and a total-element limit of
16,000,000, plus a reference-table limit of 16,000,000 entries. The
reference-table cap can be tightened independently with
`Limits::max_references`; it is checked before a decoded reference is
registered. The file layer defaults to 256 MiB compressed and decompressed
input caps.
`RObject` and `RValue` access is a supported advanced API. Their fields are
encapsulated and accessed through constructors and accessors. Enum variants
may be added in minor releases, so consumers must use wildcard match arms;
the public enums are non-exhaustive. The typed `package` and `matrix` views are
the stable convenience surface for ordinary consumers.
### Installed-code corpus observation
The scheduled `Installed Code Corpus` workflow runs an R-side oracle and the
Rust scanner as separate processes against base and stats from setup-r plus
the latest versions available in the fixed `2026-09-01` P3M snapshot for
Matrix, dplyr, rlang, and R6. The manifest records the expected
package version and the provenance records the observed version, build, R
version, platform, locale, and installed path. It is intentionally limited to
scheduled and manually dispatched runs; it is not a pull-request gate.
The R 4.6.1 profile has a measured baseline containing stored-entry,
root-kind, formals-availability, and oracle comparison counts. The R 4.5.3
profile has a measured compatibility baseline with the same aggregate counts.
Both fixed-release profiles are blocking; R-devel is observational.
An eligible oracle comparison requires a same-named, non-active runtime
closure. Stored and declared names, runtime-only names, active bindings, and
runtime kind differences are reported as independent diagnostic buckets. A
bounded Rust inspection that cannot obtain formals remains an explicit
unavailable classification; it is not treated as a guessed signature.
## Stability
Typed package-metadata views are the recommended supported surface. The `RObject`/`RValue` object model is supported as an advanced surface, with variants subject to addition; unsupported SEXPs are hard errors except for selected environment internals consumed as opaque or discarded wire data. See the [workspace stability policy](https://github.com/eitsupi/r-documentation-rs/blob/main/STABILITY.md).
## License
MIT; see [the workspace license](../../LICENSE).