kapsl-scheduler 0.1.0

Request scheduling and batching for the Kapsl inference engine
Documentation 
1
2
3
4
5

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Priority {
    LatencyCritical,
    Throughput,
}