Struct NumericFeatureBuffer

Source

pub struct NumericFeatureBuffer {
    pub feature_set_id: String,
    pub representation_id: RepresentationId,
    pub feature_names: Vec<String>,
    pub observation_ids: Vec<ObservationId>,
    /* private fields */
}

Fields§

§feature_set_id: String§representation_id: RepresentationId§feature_names: Vec<String>§observation_ids: Vec<ObservationId>

Implementations§

Source §

impl NumericFeatureBuffer

Source

pub fn from_feature_table( table: CoordinatorFeatureTable, ) -> Result<NumericFeatureBuffer, DataError>

Source

pub fn from_f64_matrix( matrix: NumericFeatureMatrixF64, ) -> Result<NumericFeatureBuffer, DataError>

Source

pub fn from_f64_column_matrix( matrix: NumericFeatureMatrixF64Columnar, ) -> Result<NumericFeatureBuffer, DataError>

Source

pub fn fingerprint(&self) -> Result<String, DataError>

Reproducibility fingerprint of the buffer’s logical content as a 64-char lowercase hex SHA-256.

The preimage is fed to the hasher incrementally — there is NO O(n_cells) intermediate byte vector or per-cell boxed value — so a multi-million-cell matrix costs one fixed scratch buffer, not a transient copy of the whole payload (the old serde_json::to_vec path).

Preimage layout (every multi-byte field little-endian; every variable-length field/collection length-prefixed so framing is unambiguous):

domain tag  b"dag-ml-data.numeric-feature-buffer.v2\0"   (fixed literal)
feature_set_id        : str   (u64 byte-len, then UTF-8)
representation_id     : str   (u64 byte-len, then UTF-8)
feature_names         : str[] (u64 count, then each str)
observation_ids       : str[] (u64 count, then each str)
n_rows                : u64   (fixed-width LE)
n_cols                : u64   (fixed-width LE)
cells row-major over (row, col):
  each cell           : 1 presence byte (1=Some, 0=None) + 8 LE f64 bytes

Rationale for the load-bearing choices:

Shape is hashed explicitly as fixed-width n_rows/n_cols, and the cell stream is emitted in one fixed traversal, so a 2x3 and a 3x2 buffer with identical flat values produce different digests.
Storage is column-major (columns[feature][row]); we deliberately traverse row-major over (row, col) and document that fixed order here. The constructors normalize both row-major and column-major typed inputs into the same columns storage, so the same logical matrix always streams in the same order regardless of input layout.
Masked cells emit a 0 presence byte then 8 zero value bytes, a fixed 9-byte stride. The presence byte alone separates None from Some(0.0) (whose value bytes are also all zero), so a mask flip is never indistinguishable from a real value.

Source

pub fn manifest(&self) -> Result<NumericFeatureBufferManifest, DataError>

Source

pub fn to_f64_column_matrix(&self) -> NumericFeatureMatrixF64Columnar

Returns the buffer’s contents as a NumericFeatureMatrixF64Columnar, suitable for binary persistence or pass-through to other typed-input constructors. The output is the inverse of NumericFeatureBuffer::from_f64_column_matrix: a round-trip through this method and the constructor preserves the buffer fingerprint.

Source

pub fn selected_indices( &self, columns: Option<&[String]>, ) -> Result<Vec<usize>, DataError>

Source

pub fn project_relations( &self, relations: &CoordinatorRelationSet, source_id: Option<&SourceId>, columns: Option<&[String]>, ) -> Result<CoordinatorFeatureBlock, DataError>

Source

pub fn project_relations_f64( &self, relations: &CoordinatorRelationSet, source_id: Option<&SourceId>, columns: Option<&[String]>, ) -> Result<CoordinatorFeatureBlockF64, DataError>

Like Self::project_relations but flattens the cells row-major straight into one Vec<f64> — no rows × cols boxed serde_json::Values. This is the hot path for large numeric blocks crossing a typed-array binding boundary. Masked (invalid) cells are an error: the typed projection has no Null to project them into.