Enum PythonBatchRef 

pub enum PythonBatchRef {
    CachedMicroshardGroup {
        shard_paths: Vec<PathBuf>,
        microshard_ids: Vec<String>,
        ordinals: Vec<u32>,
        bytes_len: u64,
    },
    MicroEpoch {
        lease_id: String,
        microshard_ids: Vec<String>,
        ordinals: Vec<u32>,
        bytes_len: u64,
        pipeline_kind: LeaseDataPipelineKind,
        payload: Value,
    },
}

One lease-loaded micro-epoch batch passed to the Python worker.

Variants

CachedMicroshardGroup

Cached shard group materialized by the Rust shard cache.

Fields

shard_paths: Vec<PathBuf>

Paths of the cached microshards that should be consumed together.

microshard_ids: Vec<String>

Stable microshard ids included in this batch group.

ordinals: Vec<u32>

Ordinals of the grouped microshards.

bytes_len: u64

Total byte size represented by the group.

MicroEpoch

Lease-scoped micro-epoch descriptor rebuilt inside Python.

Fields

lease_id: String

Lease id driving this micro-epoch.

microshard_ids: Vec<String>

Stable microshard ids included in the lease.

ordinals: Vec<u32>

Ordinals of the microshards included in the lease.

bytes_len: u64

Total byte size represented by the cached microshards, when known.

pipeline_kind: LeaseDataPipelineKind

High-level pipeline kind associated with this lease.

payload: Value

Opaque workload-specific payload consumed by Python.

Implementations

impl PythonBatchRef

pub fn cached_microshard_group(group: &[CachedMicroShard]) -> Self

Builds a shard-backed batch ref.

pub fn micro_epoch( lease: &AssignmentLease, cached_microshards: &[CachedMicroShard], pipeline_kind: LeaseDataPipelineKind, payload: Value, ) -> Self

Builds a generic micro-epoch descriptor for Python-side dataloader reconstruction.

Trait Implementations

impl Clone for PythonBatchRef

fn clone(&self) -> PythonBatchRef

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PythonBatchRef

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for PythonBatchRef

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for PythonBatchRef

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.