Struct TaskManager 

pub struct TaskManager<T: 'static + AsLogicalPlan, U: 'static + AsExecutionPlan> { /* private fields */ }

Implementations

impl<T: 'static + AsLogicalPlan, U: 'static + AsExecutionPlan> TaskManager<T, U>

pub fn new( state: Arc<dyn JobState>, codec: KapotCodec<T, U>, scheduler_id: String, ) -> Self

pub fn queue_job( &self, job_id: &str, job_name: &str, queued_at: u64, ) -> Result<()>

Enqueue a job for scheduling

pub fn pending_job_number(&self) -> usize

Get the number of queued jobs. If it’s big, then it means the scheduler is too busy. In normal case, it’s better to be 0.

pub fn running_job_number(&self) -> usize

Get the number of running jobs.

pub async fn submit_job( &self, job_id: &str, job_name: &str, session_id: &str, plan: Arc<dyn ExecutionPlan>, queued_at: u64, ) -> Result<()>

Generate an ExecutionGraph for the job and save it to the persistent state. By default, this job will be curated by the scheduler which receives it. Then we will also save it to the active execution graph

pub fn get_running_job_cache(&self) -> Arc<HashMap<String, JobInfoCache>>

pub async fn get_jobs(&self) -> Result<Vec<JobOverview>>

Get a list of active job ids

pub async fn get_job_status(&self, job_id: &str) -> Result<Option<JobStatus>>

Get the status of of a job. First look in the active cache. If no one found, then in the Active/Completed jobs, and then in Failed jobs

pub async fn fail_unscheduled_job( &self, job_id: &str, failure_reason: String, ) -> Result<()>

Mark a unscheduled job as failed. This will create a key under the FailedJobs keyspace and remove the job from ActiveJobs or QueuedJobs

pub async fn update_job(&self, job_id: &str) -> Result<usize>

pub async fn executor_lost( &self, executor_id: &str, ) -> Result<Vec<RunningTaskInfo>>

return a Vec of running tasks need to cancel

pub async fn get_available_task_count(&self, job_id: &str) -> Result<usize>

Retrieve the number of available tasks for the given job. The value returned is strictly a point-in-time snapshot

pub fn prepare_task_definition( &self, task: TaskDescription, ) -> Result<TaskDefinition>

pub fn generate_job_id(&self) -> String

Generate a new random Job ID

Trait Implementations

impl<T: Clone + 'static + AsLogicalPlan, U: Clone + 'static + AsExecutionPlan> Clone for TaskManager<T, U>

fn clone(&self) -> TaskManager<T, U>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.