Enum ArrayOp 

pub enum ArrayOp {
    OpenArray {
        array_id: ArrayId,
        schema_msgpack: Vec<u8>,
        schema_hash: u64,
        prefix_bits: u8,
    },
    Put {
        array_id: ArrayId,
        cells_msgpack: Vec<u8>,
        wal_lsn: u64,
        provenance: Option<SyncProvenance>,
    },
    Delete {
        array_id: ArrayId,
        coords_msgpack: Vec<u8>,
        wal_lsn: u64,
        provenance: Option<SyncProvenance>,
    },
    Slice {
        array_id: ArrayId,
        slice_msgpack: Vec<u8>,
        attr_projection: Vec<u32>,
        limit: u32,
        cell_filter: Option<SurrogateBitmap>,
        hilbert_range: Option<(u64, u64)>,
        system_time: SystemTimeScope,
        valid_at_ms: Option<i64>,
    },
    Project {
        array_id: ArrayId,
        attr_indices: Vec<u32>,
    },
    Aggregate {
        array_id: ArrayId,
        attr_idx: u32,
        reducer: ArrayReducer,
        group_by_dim: i32,
        cell_filter: Option<SurrogateBitmap>,
        return_partial: bool,
        hilbert_range: Option<(u64, u64)>,
        system_as_of: Option<i64>,
        valid_at_ms: Option<i64>,
    },
    Elementwise {
        left: ArrayId,
        right: ArrayId,
        op: ArrayBinaryOp,
        attr_idx: u32,
        cell_filter: Option<SurrogateBitmap>,
    },
    Flush {
        array_id: ArrayId,
        wal_lsn: u64,
    },
    Compact {
        array_id: ArrayId,
        audit_retain_ms: Option<i64>,
    },
    SurrogateBitmapScan {
        array_id: ArrayId,
        slice_msgpack: Vec<u8>,
    },
    DropArray {
        array_id: ArrayId,
    },
}

Array engine physical operations.

All ops scope by ArrayId (tenant + array name). LSN-bearing write ops (Put, Delete) carry a Control-Plane-allocated wal_lsn so the Data Plane handler can stamp memtable entries without re-appending to the WAL — matching the existing timeseries / columnar dispatch contract.

Variants

OpenArray

Open or attach to an existing array. Schema bytes are an zerompk encoding of nodedb_array::ArraySchema. The DDL pathway will replace this with a catalog lookup.

Fields

array_id: ArrayId

schema_msgpack: Vec<u8>

schema_hash: u64

prefix_bits: u8

Hilbert-prefix bits for vShard routing. Immutable post-create.

Put

Insert one or more cells. cells_msgpack is an zerompk encoding of Vec<nodedb::engine::array::wal::ArrayPutCell>. Each cell carries a Control-Plane-allocated Surrogate parallel to its coord tuple — cross-engine bitmap joins read the surrogate column on read paths instead of translating coords back to user-visible PKs.

provenance is Some only on the inbound sync path (Lite → Origin). The Data Plane uses it for epoch fencing and HWM tracking. Non-sync callers (SQL DML, Raft apply) set this to None.

Fields

array_id: ArrayId

cells_msgpack: Vec<u8>

wal_lsn: u64

provenance: Option<SyncProvenance>

Sync provenance for epoch fencing and HWM tracking. None for locally-originated or Raft-replicated writes.

Delete

Delete by exact coordinates. coords_msgpack is an zerompk encoding of Vec<Vec<CoordValue>>.

provenance is Some only on the inbound sync path (Lite → Origin).

Fields

array_id: ArrayId

coords_msgpack: Vec<u8>

wal_lsn: u64

provenance: Option<SyncProvenance>

Sync provenance for epoch fencing and HWM tracking. None for locally-originated or Raft-replicated writes.

Slice

Coord-range slice with optional attribute projection. slice_msgpack is an zerompk encoding of nodedb_array::query::Slice. Empty attr_projection means “all attributes”.

Under SystemTimeScope::AllVersions (AS OF SYSTEM TIME NULL), the Data Plane emits one row per live cell-version, each carrying ArrayCell::system_time = Some(system_from_ms). Rows are sorted ascending by system time, ties broken by coord lexicographic order. limit bounds the total number of versions returned (not cells).

Fields

array_id: ArrayId

slice_msgpack: Vec<u8>

attr_projection: Vec<u32>

limit: u32

cell_filter: Option<SurrogateBitmap>

Optional surrogate prefilter restricting which cells are returned. Only cells whose surrogate is present in this bitmap are included. None = no restriction.

hilbert_range: Option<(u64, u64)>

Optional Hilbert-prefix range [lo, hi]. When set, only cells whose Hilbert prefix falls within this range are included. Used by distributed shard slice to prevent duplicate rows in single-node harnesses where all vShards share one Data Plane. None = no Hilbert filter (all cells included).

system_time: SystemTimeScope

Bitemporal system-time scope.

• Current: live read (returns the latest committed state).
• AsOf(t): point-in-time snapshot at system-time t.
• AllVersions: audit-log read — every live cell-version, ordered ascending by system_from_ms. Each emitted row has ArrayCell::system_time = Some(system_from_ms).

valid_at_ms: Option<i64>

Bitemporal valid-time point. When Some(vt), a cell is only returned if valid_from_ms <= vt < valid_until_ms. None = no valid-time filter.

Project

Attribute projection (no coord filter).

Fields

array_id: ArrayId

attr_indices: Vec<u32>

Aggregate

Reduce one attribute column. group_by_dim < 0 means no group-by (one scalar partial per tile, merged across tiles by the executor).

Fields

array_id: ArrayId

attr_idx: u32

reducer: ArrayReducer

group_by_dim: i32

cell_filter: Option<SurrogateBitmap>

Optional surrogate prefilter: only cells whose surrogate is in this bitmap contribute to the aggregate. None = all cells.

return_partial: bool

When true the Data Plane returns a zerompk-encoded Vec<ArrayAggPartial> instead of the finalized scalar rows. Used by the distributed shard handler so the coordinator can merge partials across shards before finalizing. Single-node SQL callers always pass false.

hilbert_range: Option<(u64, u64)>

Optional Hilbert-prefix range [lo, hi]. When set, only cells whose Hilbert prefix falls within this range are included. Used by distributed shard agg to prevent double-counting in single-node harnesses where all vShards share one Data Plane. None = no Hilbert filter (all cells included).

system_as_of: Option<i64>

Bitemporal system-time cutoff. When Some(t), only tile versions with system_from_ms <= t contribute to the aggregate. None = live read.

valid_at_ms: Option<i64>

Bitemporal valid-time point. When Some(vt), only cells whose valid_from_ms <= vt < valid_until_ms contribute. None = no valid-time filter.

Elementwise

Pairwise op between two coord-aligned arrays. Both must share schema (validated on the Data Plane).

Fields

left: ArrayId

right: ArrayId

op: ArrayBinaryOp

attr_idx: u32

cell_filter: Option<SurrogateBitmap>

Optional surrogate prefilter restricting which cells participate in the pairwise op. Applied to both operands so outer-join fallthroughs from either side are also excluded. None = all cells.

Flush

Force a memtable flush. Returns the new segment ref’s id + flush_lsn in the response payload. The Control Plane allocates wal_lsn from the central WAL writer and the engine stamps it as the segment’s flush watermark.

Fields

array_id: ArrayId

wal_lsn: u64

Compact

Trigger compaction if the picker selects one. Response indicates whether a merge happened.

audit_retain_ms is the array’s retention window in milliseconds (from ArrayCatalogEntry). None means retain all versions.

Fields

array_id: ArrayId

audit_retain_ms: Option<i64>

SurrogateBitmapScan

Coord-range slice that emits one document-shaped row per matching cell, where the row’s id is the cell’s bound Surrogate formatted as 8-char zero-padded lowercase hex (substrate row-key format). Used by the cross-engine fusion path: the vector engine runs this as an inline_prefilter_plan and reads id via collect_surrogates to build a SurrogateBitmap. Since the surrogate identity initiative binds every cell to a global Surrogate at write time, the cell’s own surrogate is the cross-engine join key — no extra attr resolution needed.

Fields

array_id: ArrayId

slice_msgpack: Vec<u8>

DropArray

Drop the per-core array store. Broadcast on DROP ARRAY after the Control-Plane catalog has been mutated so each Data-Plane core releases its local store; otherwise a follow-up CREATE ARRAY of the same name could carry stale memtable / segment state. Idempotent: silently succeeds when no store is open on the receiving core.

Fields

array_id: ArrayId

Implementations

impl ArrayOp

pub fn primary_array(&self) -> &ArrayId

The array this op targets. For Elementwise, returns the left operand — vshard routing pins to the left array’s shard and the right is fetched cross-shard.

Trait Implementations

impl Clone for ArrayOp

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ArrayOp

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ArrayOp

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<'__msgpack_de> FromMessagePack<'__msgpack_de> for ArrayOp

fn read<R: Read<'__msgpack_de>>(reader: &mut R) -> Result<Self, Error>

where Self: Sized,

Reads the MessagePack representation of this value from the provided reader.

impl PartialEq for ArrayOp

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for ArrayOp

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ArrayOp

impl ToMessagePack for ArrayOp

fn write<W: Write>(&self, writer: &mut W) -> Result<(), Error>

Writes the MessagePack representation of this value into the provided writer.