opendal-core 0.56.0

- Proposal Name: core
- Start Date: 2025-11-27
- RFC PR: [apache/opendal#6828](https://github.com/apache/opendal/pull/6828)
- Tracking Issue: [apache/opendal#6829](https://github.com/apache/opendal/issues/6829)

# Summary

Introduce a crate topology where `opendal-core` owns types, raw traits, and a minimal set of critical layers; each storage backend lives in its own `opendal-service-{backend}` crate; non-critical layers move to `opendal-layer-{name}` crates; and a facade crate `opendal` re-exports core, services, and layers while wiring dynamic service registration and feature aliases.

# Motivation

- Decouple compilation units so users only build the services and layers they need, reducing compile time and dependency footprint.
- Make service delivery and versioning independent, enabling faster backend evolution without touching core.
- Allow layer evolution at its own pace while keeping correctness-critical layers close to core for safety.
- Prepare for future plugin ecosystems (e.g., third-party services/layers published independently) without breaking existing API paths.

# Guide-level explanation

As a user you depend on the facade crate `opendal` as before. Differences:

- To enable a backend, turn on the corresponding feature, for example `opendal = { version = "x.y", features = ["service-s3"] }`. The feature pulls in `opendal-service-s3`.
- Critical layers (error context, simulate, completeness, correctness check, capability check, immutable index, HTTP client) are always available via `opendal::layers` because they live in `opendal-core`.
- Non-critical layers are opt-in features, e.g., `"layer-logging"` pulls `opendal-layer-logging` and re-exports `LoggingLayer` under `opendal::layers::LoggingLayer`.
- API surface stays the same: `opendal::Operator`, `opendal::services::S3`, `opendal::layers::LoggingLayer` still work; only crate boundaries change under the hood.

Migration for existing users:

```toml
# Old
opendal = { version = "0.55", features = ["services-s3", "layers-logging"] }

# New (facade keeps aliases for one major version)
opendal = { version = "0.56", features = ["service-s3", "layer-logging"] }
```

# Reference-level explanation

## Crate layout

- `opendal-core`
  - Modules: `types`, `raw`, `blocking`, `docs`, critical layers (ErrorContext, Simulate, Complete, CorrectnessCheck, CapabilityCheck, ImmutableIndex, HttpClient, optional TailCut).
  - No service implementations, no non-critical layers.
  - Exposes `OperatorRegistry` and `Scheme` but performs no builtin registrations.
- `opendal-service-{backend}`
  - Contains the backend’s `Builder`, `Config`, accessor implementation, scheme constant, and a `register(&OperatorRegistry)` helper.
  - Depends only on `opendal-core` (plus backend-specific deps).
- `opendal-layer-{name}`
  - Each non-critical layer in its own crate (or small groups when strongly coupled, e.g., oteltrace/otelmetrics).
  - Depends on `opendal-core` and layer-specific deps.
- `opendal` facade
  - Depends on `opendal-core` (mandatory), `opendal-layer-*` and `opendal-service-*` as optional deps behind features.
  - Re-exports `core::{Operator, Result, ...}`, critical layers, and optional services/layers.
  - On build, the facade crate registers enabled services via a single `ctor` initializer to populate `DEFAULT_OPERATOR_REGISTRY`.
  - Maintains feature aliases for one major version (e.g., `services-s3` -> `service-s3`, `layers-logging` -> `layer-logging`).

## Feature matrix

- Core features: keep executor/runtime toggles (`executors-tokio`, `blocking`), internal helpers (`internal-path-cache`, `internal-tokio-rt`), plus minimal TLS/http knobs.
- Service features: `service-{backend}` live in facade, map 1:1 to `opendal-service-{backend}` optional dependency.
- Layer features: `layer-{name}` live in facade, map 1:1 to `opendal-layer-{name}` optional dependency; critical layers require no feature.

## Registry and Scheme

- `Operator::from_uri` remains, but core contains no builtin registrations.
- Facade constructs registry at init by conditionally calling each service crate’s `register` based on enabled features.
- `Scheme::enabled()` moves to facade to aggregate enabled services; core provides base enum and helper for custom schemes.

## Testing and CI

- Core tests remain for types/raw/critical layers.
- Each service crate carries unit tests plus behavior tests triggered via shared harness; a new `opendal-behavior` test crate can reuse the existing fixtures.
- Each layer crate keeps its own minimal tests; integration tests that combine layers/services run in facade or dedicated smoke suites.
- Workspace-level tasks (`cargo clippy`, `cargo fmt --check`) cover all crates; behavior matrix splits by service features to control runtime cost.

## Compatibility

- Public API paths stay stable through facade re-exports.
- Feature aliases maintained for one major release; deprecation warnings in docs and cargo feature descriptions.
- Crate names and versions remain under Apache OpenDAL namespace.

# Drawbacks

- More crates increase release coordination overhead and CI matrix size.
- Users pinning to core-only without facade must manually enable services/layers they need.
- Potential longer resolve time in dependency solvers due to many optional dependencies.

# Rationale and alternatives

- Keeping critical layers in core avoids breaking invariants and reduces dependency churn on safety-sensitive code.
- Per-layer and per-service crates maximize compile-time selectivity and allow independent evolution.
- Alternative considered: one `opendal-layers` mega-crate; rejected to keep feature granularity and lighter dependency pull for single-layer use.

# Prior art

- Rust ecosystem crates like `tokio` vs `tokio-util`, `tower` vs `tower-*` split utilities from core runtime.
- Storage SDKs (AWS SDK for Rust) split per-service crates to reduce bloat and compile time.

# Unresolved questions

- Exact set of critical layers that must stay in core (TailCut, Retry?) may need tuning after performance and safety review.
- Whether to group strongly coupled layers (otelmetrics/oteltrace) into one crate or keep fully separate.
- Behavior test harness packaging: shared crate vs duplicated fixtures.

# Future possibilities

- Allow third-party service or layer crates to self-register via cargo features or a plugin registry.
- Provide a `minimal` facade profile that ships only core + registry glue for embedders.
- Introduce codegen to auto-wire service registrations to shrink manual glue in facade.