Struct Node 

pub struct Node<T>
where
    T: Chunk,
{
    pub path: PathBuf,
    pub chunk: T,
    pub vectors: Option<HashMap<EmbeddedField, Vec<f32>>>,
    pub sparse_vectors: Option<HashMap<EmbeddedField, SparseEmbedding>>,
    pub metadata: Metadata,
    pub embed_mode: EmbedMode,
    pub original_size: usize,
    pub offset: usize,
}

Represents a unit of data in the indexing process.

Node encapsulates all necessary information for a single unit of data being processed in the indexing pipeline. It includes fields for an identifier, file path, data chunk, optional vector representation, and metadata.

Fields

path: PathBuf

File path associated with the node.

chunk: T

Data chunk contained in the node.

vectors: Option<HashMap<EmbeddedField, Vec<f32>>>

Optional vector representation of embedded data.

sparse_vectors: Option<HashMap<EmbeddedField, SparseEmbedding>>

Optional sparse vector representation of embedded data.

metadata: Metadata

Metadata associated with the node.

embed_mode: EmbedMode

Mode of embedding data Chunk and Metadata

original_size: usize

Size of the input this node was originally derived from in bytes

offset: usize

Offset of the chunk relative to the start of the input this node was originally derived from in bytes

Implementations

impl<T> Node<T>

where T: Chunk,

pub fn build_from_other(node: &Node<T>) -> NodeBuilder<T>

Builds a new instance of Node, returning a NodeBuilder. Copies over the fields from the provided Node.

pub fn builder<VALUE>() -> NodeBuilder<VALUE>

where VALUE: Chunk + Clone,

Creates a new instance of NodeBuilder.

pub fn new(chunk: impl Into<String>) -> Node<String>

Creates a new instance of Node with the specified data chunk.

The other fields are set to their default values.

pub fn with_metadata(&mut self, metadata: impl Into<Metadata>) -> &mut Node<T>

pub fn with_vectors( &mut self, vectors: impl Into<HashMap<EmbeddedField, Vec<f32>>>, ) -> &mut Node<T>

pub fn with_sparse_vectors( &mut self, sparse_vectors: impl Into<HashMap<EmbeddedField, SparseEmbedding>>, ) -> &mut Node<T>

pub fn id(&self) -> Uuid

Retrieve the identifier of the node.

Calculates the identifier of the node based on its path and chunk as bytes, returning a UUID (v3).

WARN: Does not memoize the id. Use sparingly.

impl Node<String>

pub fn as_embeddables(&self) -> Vec<(EmbeddedField, String)>

Creates embeddable data depending on chosen EmbedMode.

Returns

Embeddable data mapped to their EmbeddedField.

Trait Implementations

impl<T> Clone for Node<T>

where T: Clone + Chunk,

fn clone(&self) -> Node<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T> Debug for Node<T>

where T: Chunk,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the node for debugging purposes.

This method is used to provide a human-readable representation of the node when debugging. The vector field is displayed as the number of elements in the vector if present.

impl<T> Default for Node<T>

where T: Default + Chunk,

fn default() -> Node<T>

Returns the “default value” for a type.

impl<'de, T> Deserialize<'de> for Node<T>

where T: Chunk + Deserialize<'de>,

fn deserialize<__D>( __deserializer: __D, ) -> Result<Node<T>, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<T> From<T> for Node<String>

where T: Into<String>,

fn from(value: T) -> Node<String>

Converts to this type from the input type.

impl Hash for Node<String>

fn hash<H>(&self, state: &mut H)

where H: Hasher,

Hashes the node based on its path and chunk.

This method is used by the calculate_hash method to generate a hash value for the node.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl<T> PartialEq for Node<T>

where T: PartialEq + Chunk,

fn eq(&self, other: &Node<T>) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl<T> Serialize for Node<T>

where T: Chunk + Serialize,

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl<T> StructuralPartialEq for Node<T>

where T: PartialEq + Chunk,