# Qubit Metadata
[](https://github.com/qubit-ltd/rs-metadata/actions/workflows/ci.yml)
[](https://qubit-ltd.github.io/rs-metadata/coverage/)
[](https://crates.io/crates/qubit-metadata)
[](https://www.rust-lang.org)
[](LICENSE)
[](README.zh_CN.md)
`qubit-metadata` is a typed, ordered metadata model for Rust applications that
need extensible fields without weakening the types of their core data model.
It combines strict reads, explicit conversion, optional schemas, composable filters, and
strict Serde wire formats in one small API.
## A real use case
Suppose a document pipeline stores chunks that may later be sent to a vector
database. The chunk type can keep stable fields in its own struct and use
`Metadata` for fields that vary by source or indexing backend:
```rust
use qubit_metadata::Metadata;
let metadata = Metadata::new()
.with("tenant_id", "acme")
.with("document_id", "doc-42")
.with("chunk_index", 3_i64)
.with("language", "en");
let tenant: Option<String> = metadata.get_optional("tenant_id").unwrap();
assert_eq!(tenant.as_deref(), Some("acme"));
assert_eq!(metadata.get::<i64>("chunk_index").unwrap(), 3);
```
The stored values retain concrete runtime types through `qubit_value::Value`.
When a backend needs predictable fields and query validation, add a
`MetadataSchema` and build a `MetadataFilter` against it. The complete
scenario, including diagnostics and wire limits, is covered in the
[English user guide](doc/user_guide.md).
## Why this project exists
Plain maps are convenient, but they leave important decisions to every caller:
which values are valid, whether a field is required, how a filter is grouped,
and how untrusted serialized input is bounded. This crate centralizes those
decisions while keeping the core metadata container independent of any storage
provider or domain model.
## Installation
```toml
[dependencies]
qubit-metadata = "0.7"
```
The default feature set provides the core metadata container only. Enable
`schema` when you need schema validation; it includes `filter`:
```toml
[dependencies]
qubit-metadata = { version = "0.7", features = ["schema"] }
qubit-datatype = "0.15"
```
Optional features are `chrono`, `big-integer`, `big-decimal`, `big-number`,
`url`, `json`, and `all`. Use the direct `qubit-datatype` dependency when
declaring schema field types, and `qubit-value` when constructing `Value`
operands directly. Depend on `qubit-budget` directly when customizing the
directional JSON limit profiles.
## What it provides
- `Metadata`: ordered `String -> Value` storage with strict `get`, borrowed
`get_ref`, optional/defaulted reads, explicit `convert`, `set`, `insert`, `with`, iteration, merge, and schema-checked
writes.
- `MetadataSchema`: required/optional field definitions, concrete
`qubit_datatype::DataType` validation, and independent policies for unknown
metadata and filter fields.
- `FilterExpression` and `MetadataFilter`: immutable Boolean expressions with
equality, range, membership, existence, grouping, negation, matching options,
and receiver-side expression limits.
- Strict V1 Serde formats for metadata, schemas, and filters. The `json`
feature also provides bounded JSON-slice decoding and encoding.
- Structured `MetadataError`, validation errors, and wire-decode errors for
callers that need to distinguish missing keys, unset values, type mismatches,
invalid expressions, and input-limit failures.
Decode and encode policies are deliberately independent. The `json` feature
enables JSON round trips; configure directional budgets in the
[English user guide](doc/user_guide.md#directional-json-budgets).
The JSON round-trip example needs this dependency:
```toml
[dependencies]
qubit-metadata = { version = "0.7", features = ["json"] }
```
```rust
use qubit_metadata::Metadata;
pub fn main() -> Result<(), Box<dyn std::error::Error>> {
let metadata = Metadata::new().with("tenant_id", "acme");
let bytes = metadata.to_json_vec()?;
let decoded = Metadata::decode_json_slice(&bytes)?;
assert_eq!(decoded.get_ref::<str>("tenant_id")?, "acme");
Ok(())
}
```
For direct `qubit-budget` configuration, see the guide.
Metadata and schema bounded JSON encoding return
`Domain(MetadataError::WireLimitExceeded { kind, value, maximum })` for V1
wire-limit failures. Callers that previously matched `Json` for schema limits
should match `Domain` and inspect its structured fields; ordinary Serde
serialization errors remain `Json`.
Use bounded slice decoding for complete untrusted JSON input. Decode and encode
profiles are independent, and metadata domain limits do not replace the shared
JSON traversal budget. See the [bounded JSON guide](doc/user_guide.md#bounded-json-decoding)
for configuration and calls, and [Resource and trust boundaries](doc/design.md#resource-and-trust-boundaries)
for implementation accounting and materialization details.
## Important boundaries
- `get` returns a strict `Result<T>`; it never converts or hides errors.
`get_optional` returns `Result<Option<T>>`, preserving type errors.
Use `convert` or `convert_with` when conversion is intended. `get_or` only
defaults missing keys and appropriately typed unset values.
- `Value::Unset` records a declared type but is not a concrete value. Required
schema fields reject it. Equality, range, and membership comparisons against
a missing or unset value produce unknown. Incomparable concrete values and
failed numeric comparisons also produce unknown; values are not implicitly
converted between types. Negation preserves unknown, so `ne`, negated
comparisons, and `not_in_set` cannot bypass an incomparable type. `matches`
returns true only for a definite true result. Existence predicates remain
definite: `exists` is false and `not_exists` is true when no concrete value
is stored, so `exists(...).not()` is true. Boolean composition follows the
usual dominance rules: `false AND unknown` is `false`, while
`true OR unknown` is `true`; the remaining mixed cases stay `unknown`.
For a concrete example, see [Three-valued filter semantics](doc/user_guide.md#three-valued-filter-semantics).
- Schema validation of stored metadata remains strict about the declared
concrete field type, even though filter schema checks accept compatible
numeric representations.
- `MetadataLimits::default()` includes byte, depth, node, sequence/map, key/string/number/payload,
and metadata-domain limits. JSON decoding limits input bytes, generic JSON structure and payload, and
metadata-domain entry, schema-field, and key counts. Domain limits cannot
exceed the canonical V1 serialization limits. Filter limits are transient
receiver-side policy and are omitted from the V1 wire; the shared JSON
adapter handles generic traversal while filter seeds enforce AST and
membership limits.
- Redacted `Debug` and `Display` output is intended for diagnostics, not as a
confidentiality boundary for arbitrary user keys or error text.
- Metadata keys are plain strings. When a key crosses a module, provider, or
storage boundary, define one string constant at the owning boundary and use
it for both writes and reads; use `MetadataSchema` when the key/value contract
must be validated.
- When limits are loaded from configuration, use
`MetadataLimits::builder().build()?` so invalid domain caps are rejected at
configuration time, consistent with `FilterLimitsBuilder::build`.
## Learn more
When upgrading dependencies from 0.6 to 0.7, see
[Migrating from 0.6 to 0.7](doc/migration_0.7.md) for the verified dependency
version map and Cargo tree diagnostics. The earlier
[0.6 API migration guide](doc/migration_0.6.md) remains available for changes
introduced before 0.7.
Version 0.6 removes `try_get*`, `get_str`, and `MetadataError::MissingValue`.
Migrate converting `try_get` calls to `convert`, strict reads to `get`, optional
strict reads to `get_optional`, and borrowed text to `get_ref::<str>`.
`MetadataError::ValueAccess` retains a `ValueError`, including `ValueMissing`
facts and the original conversion error through `Error::source`. Filter and
schema behavior, numeric comparison, and Wire V1 stay unchanged. Unlike
`Config::get`, `Metadata::get` now requires the exact stored type.
- [English user guide](doc/user_guide.md)
- [Migrating from 0.6 to 0.7](doc/migration_0.7.md)
- [Migrating to 0.6](doc/migration_0.6.md)
- [中文用户手册](doc/user_guide.zh_CN.md)
- [English design](doc/design.md)
- [中文设计文档](doc/design.zh_CN.md)
- [Rust API documentation](https://docs.rs/qubit-metadata)
- [中文 README](README.zh_CN.md)
## Testing
```bash
# Run tests with the default feature set
cargo test
# Run tests with all declared features
cargo test --all-features
# Project CI checks
./.infra/bin/ci-check.sh
# Check code coverage
./.infra/bin/coverage.sh
```
## License
Copyright (c) 2025 - 2026. Haixing Hu. All rights reserved.
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the
full license text.
## Contributing
Contributions are welcome. Please follow the Rust API guidelines, keep public
API documentation and tests current, and run `./.infra/bin/align-ci.sh` to format code and
`./.infra/bin/ci-check.sh` to satisfy CI requirements before submitting a pull request.
## Author
**Haixing Hu** - *Qubit Co. Ltd.*
Repository: [https://github.com/qubit-ltd/rs-metadata](https://github.com/qubit-ltd/rs-metadata)