pub struct ComputationGraphStore { /* private fields */ }Expand description
CID-linked computation graph store.
Maintains the directed acyclic graph (DAG) of ComputationNodes that
constitutes the forward pass. After forward execution each node is linked
to its output CID; after the backward pass each node carries a gradient CID.
The graph can be checkpointed to bytes and restored for resume.
Implementations§
Source§impl ComputationGraphStore
impl ComputationGraphStore
Sourcepub fn add_node(&mut self, node: ComputationNode)
pub fn add_node(&mut self, node: ComputationNode)
Insert a node into the graph.
Edges are automatically derived from node.input_cids: for each input
CID we look for an existing node whose output_cid matches, and record
a directed edge from that producer node to the new node.
Sourcepub fn record_output(
&mut self,
node_id: &str,
output_cid: String,
) -> Result<(), GradientError>
pub fn record_output( &mut self, node_id: &str, output_cid: String, ) -> Result<(), GradientError>
Record that the forward pass for node_id produced output_cid.
After recording, edges to any downstream nodes that list this CID as an input are added automatically.
Sourcepub fn record_gradient(
&mut self,
node_id: &str,
grad_cid: String,
) -> Result<(), GradientError>
pub fn record_gradient( &mut self, node_id: &str, grad_cid: String, ) -> Result<(), GradientError>
Record that the backward pass computed a gradient for node_id.
Sourcepub fn topological_order(&self) -> Vec<String>
pub fn topological_order(&self) -> Vec<String>
Return node IDs in topological order (Kahn’s algorithm).
Nodes with no predecessors come first; this is the correct execution order for the forward pass (and reverse for backward).
Sourcepub fn store_gradient_as_arrow(
&mut self,
node_id: &str,
gradient_data: &[f32],
shape: &[usize],
) -> Result<String, GradientError>
pub fn store_gradient_as_arrow( &mut self, node_id: &str, gradient_data: &[f32], shape: &[usize], ) -> Result<String, GradientError>
Serialize a gradient tensor as an Apache Arrow IPC block and store its
CID in the named node’s gradient_cid field.
The gradient data is wrapped in an crate::arrow::ArrowTensorStore
as a single f32 column named "gradient". Shape metadata is embedded
in the Arrow field metadata so it survives a round-trip through
Self::load_gradient_from_arrow.
The CID is computed as a DAG-CBOR CID (SHA-256) over the raw Arrow IPC
bytes, matching the convention used throughout ipld_codec.
§Returns
The CID string that was recorded on the node.
Sourcepub fn load_gradient_from_arrow(
arrow_bytes: &[u8],
expected_shape: &[usize],
) -> Result<Vec<f32>, GradientError>
pub fn load_gradient_from_arrow( arrow_bytes: &[u8], expected_shape: &[usize], ) -> Result<Vec<f32>, GradientError>
Decode gradient data from raw Arrow IPC block bytes.
Validates that the decoded tensor shape matches expected_shape before
returning the f32 values. Pass an empty expected_shape slice to skip
shape validation.
Sourcepub fn checkpoint(&self) -> Result<Vec<u8>, GradientError>
pub fn checkpoint(&self) -> Result<Vec<u8>, GradientError>
Serialize the entire graph to JSON bytes for checkpointing.
Sourcepub fn from_checkpoint(data: &[u8]) -> Result<Self, GradientError>
pub fn from_checkpoint(data: &[u8]) -> Result<Self, GradientError>
Restore a graph from checkpoint bytes produced by Self::checkpoint.
Sourcepub fn find_consumers(&self, cid: &str) -> Vec<&ComputationNode>
pub fn find_consumers(&self, cid: &str) -> Vec<&ComputationNode>
Return all nodes that list cid as one of their input CIDs.
Sourcepub fn provenance_chain(&self, output_cid: &str) -> Vec<&ComputationNode>
pub fn provenance_chain(&self, output_cid: &str) -> Vec<&ComputationNode>
Return the chain of nodes whose output_cid led to output_cid being
produced, ordered from earliest producer to the node that emitted it.
This performs a depth-first traversal of the reverse DAG starting from
the node that owns output_cid.
Sourcepub fn get_node(&self, node_id: &str) -> Option<&ComputationNode>
pub fn get_node(&self, node_id: &str) -> Option<&ComputationNode>
Immutable access to a node by id.
Sourcepub fn node_count(&self) -> usize
pub fn node_count(&self) -> usize
Number of nodes currently in the graph.
Sourcepub fn edge_count(&self) -> usize
pub fn edge_count(&self) -> usize
Number of edges currently in the graph.
Sourcepub fn nodes(&self) -> impl Iterator<Item = &ComputationNode>
pub fn nodes(&self) -> impl Iterator<Item = &ComputationNode>
Iterate over all nodes (order is unspecified).
Trait Implementations§
Source§impl Clone for ComputationGraphStore
impl Clone for ComputationGraphStore
Source§fn clone(&self) -> ComputationGraphStore
fn clone(&self) -> ComputationGraphStore
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for ComputationGraphStore
impl Debug for ComputationGraphStore
Source§impl Default for ComputationGraphStore
impl Default for ComputationGraphStore
Source§fn default() -> ComputationGraphStore
fn default() -> ComputationGraphStore
Source§impl<'de> Deserialize<'de> for ComputationGraphStore
impl<'de> Deserialize<'de> for ComputationGraphStore
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Auto Trait Implementations§
impl Freeze for ComputationGraphStore
impl RefUnwindSafe for ComputationGraphStore
impl Send for ComputationGraphStore
impl Sync for ComputationGraphStore
impl Unpin for ComputationGraphStore
impl UnsafeUnpin for ComputationGraphStore
impl UnwindSafe for ComputationGraphStore
Blanket Implementations§
impl<T> Allocation for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more