Crate dag_ml_data 

Public Rust facade for dag-ml-data.

Modules

adapter

aggregation

ADR-07 canonical prediction-aggregation reducer contract.

alignment

buffer

buffer_file_store

Deterministic binary persistence for NumericFeatureBufferStore.

collation

coordinator

error

fingerprint

fitted_adapter

fusion

handle

ids

model

nd_tensor

Borrowed N-D tensor transport: host-owned dense tensors (e.g. uint8 [N,H,W,C] RGB, float32 [N,H,W,Wavelength] hyperspectral) copied into provider-owned, canonical row-major storage, keyed and gathered by observation_id.

plan

planner

relation

Structs

AdapterPath

AdapterRegistry

AdapterRegistrySpec

AdapterSpec

AggregationPolicy

Aggregation policy: which reducer collapses repeated predictions to one value per sample, plus the single parameter that reducer accepts. Parameters that do not belong to the chosen reducer must be absent (enforced by validate). None numeric parameters take the ADR-07 defaults (DEFAULT_TRIM_FRACTION, DEFAULT_THRESHOLD).

AlignmentPolicy

AugmentationMetadata

AxisSizeContract

AxisSpec

CollationPolicy

CoordinateSpec

Typed coordinate contract for an axis, so methods can machine-rely on the coordinate dtype, ordering and (for numeric) regular-grid structure rather than re-deriving them from untyped JSON.

CoordinatorBranchView

CoordinatorBranchViewSelector

CoordinatorDataHandleRecord

CoordinatorDataMaterializationRequest

CoordinatorDataPlanEnvelope

CoordinatorDataViewRecord

CoordinatorFeatureBlock

CoordinatorFeatureBlockF64

Typed sibling of CoordinatorFeatureBlock: the same projection with the cells flattened row-major into one Vec<f64> instead of per-cell JSON values. Produced by the *_f64 projection path so a large numeric block never allocates rows × cols boxed serde_json::Values. Masked cells are rejected by that path (it has no Null to project them into).

CoordinatorFeatureRow

CoordinatorFeatureTable

CoordinatorHandleArena

CoordinatorHandleRef

CoordinatorMultiTargetBlock

CoordinatorRelation

CoordinatorRelationSet

CoordinatorTargetBlock

CoordinatorTargetTable

CoordinatorTargetValue

DataErrorDescriptor

A stable ADR-11 error payload that can be serialized across bindings.

DataPlan

DataPlanRequest

DataPlanStep

DataView

DatasetSchema

FeatureFusionPolicy

FittedAdapterHandleRecord

FittedAdapterManifest

FittedAdapterManifestEntry

FittedAdapterMaterializationRequest

FittedAdapterRef

FoldAssignment

FoldSet

Exhaustive partition-style fold assignments.

FoldSpec

GroupId

GroupSpec

InMemoryFittedAdapterStore

InputPortSpec

MetadataFieldSpec

MetadataSchema

ModelInputSpec

NdTensor

A stored, immutable, contiguous row-major N-D tensor.

NdTensorArena

A store plus per-data-handle bindings, mirroring NumericFeatureBufferArena.

NdTensorBinding

Binding of one stored tensor to a materialized data handle.

NdTensorBlock

A relation-ordered, axis-0-filtered export of a stored tensor. shape[0] is the number of selected rows; data is contiguous row-major bytes.

NdTensorInput

A canonical (contiguous row-major) N-D tensor handed to the store. The C ABI layer copies any strided borrowed input into this shape before calling in.

NdTensorManifest

Provider-wide manifest of one stored tensor (no payload bytes).

NdTensorStore

An immutable collection of N-D tensors keyed by tensor_id.

NumericCollationInputBlock

NumericFeatureBuffer

NumericFeatureBufferArena

NumericFeatureBufferBinding

NumericFeatureBufferManifest

NumericFeatureBufferStore

NumericFeatureMatrixF64

NumericFeatureMatrixF64Columnar

NumericTensorBlock

ObservationId

OriginId

PathResolution

PlanIssue

PlanningPolicy

PresenceMask

RepetitionId

RepresentationId

RepresentationNode

RepresentationSpec

SampleAlignmentPlan

SampleId

SampleRelation

SampleRelationTable

ShapeContract

SourceDescriptor

SourceFeatureBlock

SourceId

SourceSampleSet

TargetId

TypeId

Enums

AggregationReducerName

Canonical per-sample aggregation reducer name (ADR-07).

AlignmentMode

AxisKind

CollationPadding

CoordinateDType

Element dtype of an axis coordinate sequence.

CoordinateValues

Axis coordinate values: either an explicit per-index list or a regular numeric grid (value(i) = start + i * step, count taken from the axis size).

CoordinatorBranchViewMode

CoordinatorHandleKind

DataError

DataPlanStepKind

FitScope

FittedAdapterBackend

GroupKind

MetadataValueKind

NdTensorDType

Element dtype of an N-D tensor. The C ABI exposes a matching #[repr(C)] DagMlDataTensorDType enum and maps to/from this type explicitly; this core enum carries no ABI guarantee of its own.

PredictionTaskKind

Prediction task kind needed to validate task-sensitive reducers. This is a deliberately tiny enum; the bridge maps its richer task type into it. It does not make dag-ml-data the home of DAG-ML task semantics.

SignalKind

SourceGranularity

Constants

CANONICAL_AGGREGATION_REDUCERS

The canonical reducer names. The custom escape hatch is intentionally not listed; it is addressed by custom_reducer_id instead. This array is the source of truth for the canonical set (the conformance pack pins only the C ABI validator symbol, not a shared reducer list).

COORDINATOR_BRANCH_VIEW_SCHEMA_ID

COORDINATOR_BRANCH_VIEW_SCHEMA_VERSION

COORDINATOR_DATA_PLAN_ENVELOPE_SCHEMA_VERSION

DEFAULT_THRESHOLD

Default threshold for exclude_outliers when unspecified (ADR-07).

DEFAULT_TRIM_FRACTION

Default trim_fraction for robust_mean when unspecified (ADR-07).

FITTED_ADAPTER_MANIFEST_SCHEMA_VERSION

FITTED_ADAPTER_REF_SCHEMA_ID

FITTED_ADAPTER_REF_SCHEMA_VERSION

ND_TENSOR_MANIFEST_SCHEMA_VERSION

Schema version of NdTensorManifest / NdTensorBinding.

ND_TENSOR_MAX_RANK

Maximum supported tensor rank. Bounds the shape-product arithmetic and keeps the contract finite.

NUMERIC_FEATURE_BUFFER_FILE_FORMAT_VERSION

NUMERIC_FEATURE_BUFFER_MANIFEST_SCHEMA_VERSION

Traits

RuntimeFittedAdapterStore

Runtime contract for materializing opaque host-owned fitted-adapter handles from a previously registered FittedAdapterRef. The store validates the request’s params_fingerprint against the registered ref and returns the opaque handle id. It never reads, writes or deserializes adapter payloads — payload materialization happens host-side, behind the opaque handle.

Functions

alignment_metadata

alignment_mode_from_fusion

build_sample_alignment_plan

collate_feature_block

collate_numeric_block

coordinator_relations_from_sample_table

data_plan_fingerprint

deserialize_columnar_store

Parse a .n4d byte stream back into a store. Rejects unknown magic, unsupported versions, truncated streams, and any tampering of the payload caught by SHA-256 mismatch.

fold_set_fingerprint

fuse_feature_blocks

numeric_input_from_feature_block

plan_model_input

read_store_from_path

Read a store back from path. Errors propagate the validation failure from deserialize_columnar_store alongside the path for diagnostics.

require_signal_type_match

Validate that a provider-declared actual signal type matches the expected one the plan or bundle records (ADR-06). The Unknown policy is the caller’s: pass allow_unknown = true at train time (an untagged signal type is tolerated) and false at predict time (a trained pipeline must carry a concrete signal type, so any difference — including Unknown — is refused).

sample_relation_fingerprint

schema_fingerprint

serialize_columnar_store

Serialize a store to its deterministic .n4d byte stream. The output ends with a 32-byte SHA-256 of all preceding bytes so callers can distinguish truncation from intentional empty stores and detect any later tampering through deserialize_columnar_store.

source_sample_set_from_feature_block

write_store_to_path

Write a store to path. Replaces any existing file at the destination.

Type Aliases

Result