Crate dag_ml_data 

Public Rust facade for dag-ml-data.

Modules

adapter

aggregation

ADR-07 canonical prediction-aggregation reducer contract.

alignment

buffer

buffer_file_store

Deterministic binary persistence for NumericFeatureBufferStore.

builtin_models

Canonical built-in data-model declarations.

collation

coordinator

error

fingerprint

fitted_adapter

fusion

handle

ids

model

nd_tensor

Borrowed N-D tensor transport: host-owned dense tensors (e.g. uint8 [N,H,W,C] RGB, float32 [N,H,W,Wavelength] hyperspectral) copied into provider-owned, canonical row-major storage, keyed and gathered by observation_id.

plan

planner

relation

representation_registry

Frozen, published catalogue of the built-in representation IDs (B-014 / DMD-001).

Structs

AdapterPath

AdapterRegistry

AdapterRegistrySpec

AdapterSpec

AggregationPolicy

Aggregation policy: which reducer collapses repeated predictions to one value per sample, plus the single parameter that reducer accepts. Parameters that do not belong to the chosen reducer must be absent (enforced by validate). None numeric parameters take the ADR-07 defaults (DEFAULT_TRIM_FRACTION, DEFAULT_THRESHOLD).

AlignmentPolicy

AugmentationMetadata

AxisSizeContract

AxisSpec

BuiltinDataModelSpec

CollationPolicy

CoordinateSpec

Typed coordinate contract for an axis, so methods can machine-rely on the coordinate dtype, ordering and (for numeric) regular-grid structure rather than re-deriving them from untyped JSON.

CoordinatorBranchView

CoordinatorBranchViewSelector

CoordinatorDataHandleRecord

CoordinatorDataMaterializationRequest

CoordinatorDataPlanEnvelope

CoordinatorDataViewRecord

CoordinatorFeatureBlock

CoordinatorFeatureBlockF64

Typed sibling of CoordinatorFeatureBlock: the same projection with the cells flattened row-major into one Vec<f64> instead of per-cell JSON values. Produced by the *_f64 projection path so a large numeric block never allocates rows × cols boxed serde_json::Values. Masked cells are rejected by that path (it has no Null to project them into).

CoordinatorFeatureRow

CoordinatorFeatureTable

CoordinatorHandleArena

CoordinatorHandleRef

CoordinatorMultiTargetBlock

CoordinatorRelation

CoordinatorRelationSet

CoordinatorTargetBlock

CoordinatorTargetTable

CoordinatorTargetValue

DataErrorDescriptor

A stable ADR-11 error payload that can be serialized across bindings.

DataPlan

DataPlanRequest

DataPlanStep

DataView

DatasetSchema

FeatureFusionPolicy

FeatureFusionSourceLayout

FittedAdapterHandleRecord

FittedAdapterManifest

FittedAdapterManifestEntry

FittedAdapterMaterializationRequest

FittedAdapterRef

FoldAssignment

FoldSet

Exhaustive partition-style fold assignments.

FoldSpec

GroupId

GroupSpec

InMemoryFittedAdapterStore

InputPortSpec

MetadataFieldSpec

MetadataSchema

ModelInputSpec

MvpStatus

MVP profile membership for a published representation ID.

NdTensor

A stored, immutable, contiguous row-major N-D tensor.

NdTensorArena

A store plus per-data-handle bindings, mirroring NumericFeatureBufferArena.

NdTensorBinding

Binding of one stored tensor to a materialized data handle.

NdTensorBlock

A relation-ordered, axis-0-filtered export of a stored tensor. shape[0] is the number of selected rows; data is contiguous row-major bytes.

NdTensorInput

A canonical (contiguous row-major) N-D tensor handed to the store. The C ABI layer copies any strided borrowed input into this shape before calling in.

NdTensorManifest

Provider-wide manifest of one stored tensor (no payload bytes).

NdTensorStore

An immutable collection of N-D tensors keyed by tensor_id.

NumericCollationInputBlock

NumericFeatureBuffer

NumericFeatureBufferArena

NumericFeatureBufferBinding

NumericFeatureBufferManifest

NumericFeatureBufferStore

NumericFeatureMatrixF64

NumericFeatureMatrixF64Columnar

NumericTensorBlock

ObservationId

OriginId

PathResolution

PlanIssue

PlanningPolicy

PresenceMask

RegisteredRepresentation

One frozen, published entry of the built-in representation registry.

RepetitionId

RepresentationId

RepresentationNode

RepresentationRegistry

The frozen, published catalogue of built-in representation IDs.

RepresentationSpec

SampleAlignmentPlan

SampleId

SampleRelation

SampleRelationTable

ShapeContract

SourceConcatLayout

SourceDescriptor

SourceFeatureBlock

SourceFeatureLayoutBlock

SourceId

SourcePreprocessingOutput

SourceSampleSet

TargetId

TypeId

Enums

AggregationReducerName

Canonical per-sample aggregation reducer name (ADR-07).

AlignmentMode

AxisKind

BuiltinDataModel

CollationPadding

CoordinateDType

Element dtype of an axis coordinate sequence.

CoordinateValues

Axis coordinate values: either an explicit per-index list or a regular numeric grid (value(i) = start + i * step, count taken from the axis size).

CoordinatorBranchViewMode

CoordinatorHandleKind

DataError

DataPlanStepKind

FitScope

FittedAdapterBackend

GroupKind

MetadataValueKind

MvpEmissionStatus

Cross-repo publication status of a representation ID inside an MVP profile.

NdTensorDType

Element dtype of an N-D tensor. The C ABI exposes a matching #[repr(C)] DagMlDataTensorDType enum and maps to/from this type explicitly; this core enum carries no ABI guarantee of its own.

PredictionTaskKind

Prediction task kind needed to validate task-sensitive reducers. This is a deliberately tiny enum; the bridge maps its richer task type into it. It does not make dag-ml-data the home of DAG-ML task semantics.

SignalKind

SourceConcatAxis

SourceFeatureLayoutKind

SourceGranularity

Constants

BUILTIN_DATA_MODELS

CANONICAL_AGGREGATION_REDUCERS

The canonical reducer names. The custom escape hatch is intentionally not listed; it is addressed by custom_reducer_id instead. This array is the source of truth for the canonical set (the conformance pack pins only the C ABI validator symbol, not a shared reducer list).

COORDINATOR_BRANCH_VIEW_SCHEMA_ID

COORDINATOR_BRANCH_VIEW_SCHEMA_VERSION

COORDINATOR_DATA_PLAN_ENVELOPE_SCHEMA_VERSION

DEFAULT_THRESHOLD

Default threshold for exclude_outliers when unspecified (ADR-07).

DEFAULT_TRIM_FRACTION

Default trim_fraction for robust_mean when unspecified (ADR-07).

FITTED_ADAPTER_MANIFEST_SCHEMA_VERSION

FITTED_ADAPTER_REF_SCHEMA_ID

FITTED_ADAPTER_REF_SCHEMA_VERSION

MVP_PROFILE_SPECTRA_IMAGE

Name of the spectra+image MVP profile defined by B-014 / IO_spec.md §5.

ND_TENSOR_MANIFEST_SCHEMA_VERSION

Schema version of NdTensorManifest / NdTensorBinding.

ND_TENSOR_MAX_RANK

Maximum supported tensor rank. Bounds the shape-product arithmetic and keeps the contract finite.

NUMERIC_FEATURE_BUFFER_FILE_FORMAT_VERSION

NUMERIC_FEATURE_BUFFER_MANIFEST_SCHEMA_VERSION

REPRESENTATION_CLIMATE_SERIES_MV

REPRESENTATION_CUBE_HWB

REPRESENTATION_DOSAGE_MATRIX

REPRESENTATION_FEATURE_BLOCK_SET

REPRESENTATION_FTIR_SIGNAL

REPRESENTATION_GRAY_IMAGE

REPRESENTATION_HYPERSPECTRAL_CUBE

REPRESENTATION_MASS_SPECTRUM

REPRESENTATION_MC_IMAGE

REPRESENTATION_MULTISPECTRAL_IMAGE

REPRESENTATION_RAMAN_SIGNAL

REPRESENTATION_REGISTRY_ID

Stable identifier of the representation registry contract.

REPRESENTATION_REGISTRY_SCHEMA_VERSION

Wire-shape version of the published representation registry manifest.

REPRESENTATION_REGISTRY_SOURCE

Repo-relative provenance of the frozen representation IDs.

REPRESENTATION_RGB_IMAGE

REPRESENTATION_ROI_MASK

REPRESENTATION_SAMPLE_METADATA

REPRESENTATION_SEGMENTATION_MASK

REPRESENTATION_SERIES_MV

REPRESENTATION_SIGNAL_1D

REPRESENTATION_SIGNAL_WITH_PROCESSINGS

REPRESENTATION_TABULAR_MIXED

REPRESENTATION_TABULAR_NUMERIC

REPRESENTATION_TARGET_CATEGORICAL

REPRESENTATION_TARGET_CATEGORICAL_MATRIX

REPRESENTATION_TARGET_NUMERIC

REPRESENTATION_TARGET_NUMERIC_MATRIX

REPRESENTATION_TEXT_RAW

REPRESENTATION_TEXT_TOKEN_IDS

REPRESENTATION_VARIANT_MATRIX

TYPE_DENSE_SIGNAL

TYPE_GENOTYPE_MATRIX

TYPE_GRAY_IMAGE

TYPE_HYPERSPECTRAL_CUBE

TYPE_IMAGE_RGB

TYPE_LABEL_MASK

TYPE_MASS_SPEC

TYPE_METADATA

TYPE_MULTICHANNEL_IMAGE

TYPE_MULTI_BLOCK

TYPE_TABLE

TYPE_TARGET

TYPE_TEXT

TYPE_TIME_SERIES

Traits

RuntimeFittedAdapterStore

Runtime contract for materializing opaque host-owned fitted-adapter handles from a previously registered FittedAdapterRef. The store validates the request’s params_fingerprint against the registered ref and returns the opaque handle id. It never reads, writes or deserializes adapter payloads — payload materialization happens host-side, behind the opaque handle.

Functions

alignment_metadata

alignment_mode_from_fusion

build_sample_alignment_plan

builtin_adapter_registry_spec

builtin_data_model_specs

builtin_representations

collate_feature_block

collate_numeric_block

coordinator_relations_from_sample_table

data_plan_fingerprint

default_builtin_planning_policy

deserialize_columnar_store

Parse a .n4d byte stream back into a store. Rejects unknown magic, unsupported versions, truncated streams, and any tampering of the payload caught by SHA-256 mismatch.

feature_block_set

fold_set_fingerprint

ftir_signal

fuse_feature_blocks

genotype_dosage_matrix

genotype_variant_matrix

gray_image

hyperspectral_cube

mass_spectrum

multichannel_image

multispectral_image

numeric_input_from_feature_block

plan_model_input

raman_signal

read_store_from_path

Read a store back from path. Errors propagate the validation failure from deserialize_columnar_store alongside the path for diagnostics.

representation_registry

Build the frozen, published catalogue of built-in representation IDs from BUILTIN_DATA_MODELS. The result is deterministic and source-order stable.

require_signal_type_match

Validate that a provider-declared actual signal type matches the expected one the plan or bundle records (ADR-06). The Unknown policy is the caller’s: pass allow_unknown = true at train time (an untagged signal type is tolerated) and false at predict time (a trained pipeline must carry a concrete signal type, so any difference — including Unknown — is refused).

rgb_image

roi_mask

sample_metadata

sample_relation_fingerprint

schema_fingerprint

segmentation_mask

serialize_columnar_store

Serialize a store to its deterministic .n4d byte stream. The output ends with a 32-byte SHA-256 of all preceding bytes so callers can distinguish truncation from intentional empty stores and detect any later tampering through deserialize_columnar_store.

series_mv

signal_1d

signal_with_processings

source_sample_set_from_feature_block

tabular_mixed

tabular_numeric

tabular_numeric_model_input_spec

target_categorical

target_categorical_matrix

target_numeric

target_numeric_matrix

text_raw

text_token_ids

validate_model_input_spec_against_registry

Validate that a model-input/data-requirements contract consumes only published representation IDs, and that each accepted representation’s frozen type_id and rank are allowed by the port.

write_store_to_path

Write a store to path. Replaces any existing file at the destination.

Type Aliases

Result