Struct BatchQueue 

pub struct BatchQueue<D>
where
    D: Clone + Send + Sync + Serialize + 'static,
{ /* private fields */ }

In-memory batch queue with model-aware reordering and ETA estimation.

The queue automatically groups jobs by resource_key to minimize expensive resource swaps (e.g. GPU model loads). It also tracks per-item processing durations bucketed by size for accurate ETA predictions.

Implementations

impl<D> BatchQueue<D>

where D: Clone + Send + Sync + Serialize + 'static,

pub fn new() -> Self

Create a new empty batch queue.

pub fn with_scheduling(scheduling: SchedulingConfig) -> Self

Create a new queue with explicit scheduling controls.

pub fn enqueue(&self, job: BatchJob<D>) -> Result<String>

Add a new batch job and perform resource-aware reordering. Returns the assigned job ID.

pub fn next_queued(&self) -> Option<BatchJob<D>>

Get the next queued job (without removing it).

pub fn mark_running(&self, job_id: &str) -> Result<()>

Mark a job as running and set its started_at timestamp.

Both the job status and scheduling metadata are updated while holding the jobs lock, ensuring they cannot become inconsistent (ABQ-2 fix: previously the jobs lock was dropped before the scheduling locks were acquired).

pub fn update_item( &self, job_id: &str, item_id: &str, status: BatchItemStatus, error: Option<String>, duration_ms: Option<u64>, ) -> Result<()>

Update a single item’s status within a job.

If the item completed successfully and duration_ms is provided, the ETA tracker is automatically updated with the new data point.

pub fn mark_completed( &self, job_id: &str, ) -> Result<Option<BatchCompletionSummary>>

Mark a job as completed and produce a completion summary.

Automatically determines whether it’s Completed or CompletedWithErrors based on item statuses.

pub fn cancel_item(&self, job_id: &str, item_id: &str) -> Result<()>

Cancel a single pending item within a job.

pub fn cancel_job(&self, job_id: &str) -> Result<()>

Cancel an entire batch job. Running items finish; pending items are cancelled.

pub fn retry_failed(&self, job_id: &str) -> Result<()>

Retry all failed items in a completed job by resetting them to Pending. The job is re-queued and reordering is applied.

Retry lineage: each retried item keeps its existing attempt_id (same logical retry family) but clears its trial_id so the executor will mint a fresh one on the next execution. This keeps batch-item retries distinguishable from outer job-level retries.

pub fn list_jobs(&self) -> Vec<BatchJob<D>>

Get all jobs (cloned snapshot).

pub fn get_job(&self, job_id: &str) -> Option<BatchJob<D>>

Get a specific job by ID.

pub fn estimate_remaining_ms(&self, job_id: &str) -> Option<u64>

Estimate remaining processing time for a job in milliseconds. Returns None if no historical data is available.

pub fn estimate_remaining(&self, job_id: &str) -> Option<EtaEstimate>

Estimate remaining processing time with confidence and sample metadata.

pub fn has_running_job(&self) -> bool

Check if any batch job is currently running.

pub fn eta_sample_count( &self, resource_key: &str, operation: &str, size_bucket: SizeBucket, ) -> u64

Get the number of ETA samples for a specific resource/operation/size combination.

pub fn queued_count(&self) -> usize

Get the number of queued (waiting) jobs.

Trait Implementations

impl<D> Default for BatchQueue<D>

where D: Clone + Send + Sync + Serialize + 'static,

fn default() -> Self

Returns the “default value” for a type.