Qubit Metadata

A general-purpose, type-safe metadata model for Rust.

Overview

qubit-metadata provides a Metadata type for attaching explicitly typed extension fields to data objects without hard-coding every auxiliary field into the core model. Typical uses include:

• Document ingestion: keep file_id, chunk_index, language, source, and confidence with each document chunk.
• Vector search: store fields such as tenant_id, doc_type, created_at, score, or acl_group that later become vector-database metadata columns or payload filters.
• Event and message pipelines: carry trace_id, request_id, tenant_id, route, and retry metadata through services.
• External service integration: retain compact service result fields such as model version, latency, billing tag, and request ID for diagnostics and analytics.

Metadata stores values as qubit_value::Value, so scalar types remain clear: i64 is different from u32, and f64 is different from String. If a caller really needs nested data, it can store Value::Json explicitly. Common document metadata, vector-database metadata, and request context are usually flat typed field sets, which keeps schema validation and filter construction predictable.

Design Goals

• Typed values: preserve concrete runtime types with qubit_value::Value.
• Convenient construction: support both mutable set() and fluent with().
• Optional schema: validate field names, required fields, concrete DataTypes, and filter compatibility.
• Filter builder: build immutable MetadataFilter values with a chainable builder.
• Serde support: serialize metadata, schemas, and filters for config, storage, and service boundaries.

Features

1) Typed metadata storage

Metadata is an ordered String -> Value map. It supports typed get, set, try_get, try_set, with, iteration, merge, retain, and conversion to/from BTreeMap<String, Value>.

use qubit_metadata::Metadata;

let meta = Metadata::new()
    .with("author", "alice")
    .with("priority", 3_i64)
    .with("reviewed", true);

assert_eq!(meta.get::<String>("author").as_deref(), Some("alice"));
assert_eq!(meta.try_get::<i64>("priority").unwrap(), 3);

2) Schema for validation and storage planning

MetadataSchema uses qubit_common::DataType. This is useful when a storage backend requires metadata fields to be declared in advance, and it also lets the filter builder validate field/operator compatibility early.

use qubit_common::DataType;
use qubit_metadata::{Metadata, MetadataSchema};

let schema = MetadataSchema::builder()
    .required("tenant_id", DataType::String)
    .required("score", DataType::Int64)
    .optional("source", DataType::String)
    .build();

let meta = Metadata::new()
    .with("tenant_id", "acme")
    .with("score", 42_i64);

schema.validate(&meta).unwrap();

3) Immutable filters built by builder

MetadataFilter::builder() creates a builder. Calling build() returns an immutable filter. build_checked(&schema) also validates referenced fields, operator compatibility, and filter value types.

use qubit_common::DataType;
use qubit_metadata::{Metadata, MetadataFilter, MetadataSchema};

let schema = MetadataSchema::builder()
    .required("status", DataType::String)
    .required("score", DataType::Int64)
    .build();

let filter = MetadataFilter::builder()
    .eq("status", "active")
    .and_ge("score", 10)
    .build_checked(&schema)
    .unwrap();

let meta = Metadata::new()
    .with("status", "active")
    .with("score", 42_i64);

assert!(filter.matches(&meta));

4) Filter DSL

Grouped subexpressions are built with closures. The closure receives a fresh builder, and the resulting expression is appended as one grouped child:

use qubit_metadata::{Metadata, MetadataFilter};

let filter = MetadataFilter::builder()
    .eq("status", "active")
    .and(|g| g.ge("score", 80).or_eq("tag", "rust"))
    .build();

let meta = Metadata::new()
    .with("status", "active")
    .with("score", 42_i64)
    .with("tag", "rust");

assert!(filter.matches(&meta));

The expression above means:

status == "active" AND (score >= 80 OR tag == "rust")

Use and_not or or_not when the whole group should be negated:

let filter = MetadataFilter::builder()
    .eq("status", "active")
    .and_not(|g| g.ge("score", 80).or_eq("tag", "java"))
    .build();

Missing-key behavior for negative predicates is controlled by MissingKeyPolicy. Mixed numeric comparison behavior is controlled by NumberComparisonPolicy.

Error Handling

Use try_get and schema validation when the caller needs diagnostics instead of Option:

use qubit_common::DataType;
use qubit_metadata::{Metadata, MetadataError};

let meta = Metadata::new().with("answer", "forty-two");

match meta.try_get::<i64>("answer") {
    Err(MetadataError::TypeMismatch { expected, actual, .. }) => {
        assert_eq!(expected, DataType::Int64);
        assert_eq!(actual, DataType::String);
    }
    other => panic!("unexpected result: {other:?}"),
}

Installation

Add to your Cargo.toml:

[dependencies]
qubit-metadata = "0.3.0"

License

Licensed under the Apache License, Version 2.0.