pub struct Batched<Inf: Inferer> { /* private fields */ }Expand description
Wraps an inferer in a batching interface. This’ll separate the data-insertion and execution, which generally improves performance.
Can be easily constructed using InfererExt::into_batched.
Implementations§
Source§impl<Inf> Batched<Inf>where
Inf: Inferer,
impl<Inf> Batched<Inf>where
Inf: Inferer,
Sourcepub fn push(&mut self, id: u64, state: State<'_>) -> Result<()>
pub fn push(&mut self, id: u64, state: State<'_>) -> Result<()>
Insert a single element into the batch to include in the next execution.
Sourcepub fn extend<'a, Iter: IntoIterator<Item = (u64, State<'a>)>>(
&mut self,
states: Iter,
) -> Result<()>
pub fn extend<'a, Iter: IntoIterator<Item = (u64, State<'a>)>>( &mut self, states: Iter, ) -> Result<()>
Insert a sequence of elements into the batch to include in the next execution.
Sourcepub fn execute(&mut self) -> Result<HashMap<u64, Response<'_>>>
pub fn execute(&mut self) -> Result<HashMap<u64, Response<'_>>>
Execute the model on the data that has been enqueued previously.
Sourcepub fn into_parts(self) -> (Inf, Batcher)
pub fn into_parts(self) -> (Inf, Batcher)
Split the batcher and the inferer.
Auto Trait Implementations§
impl<Inf> Freeze for Batched<Inf>where
Inf: Freeze,
impl<Inf> RefUnwindSafe for Batched<Inf>where
Inf: RefUnwindSafe,
impl<Inf> Send for Batched<Inf>where
Inf: Send,
impl<Inf> Sync for Batched<Inf>where
Inf: Sync,
impl<Inf> Unpin for Batched<Inf>where
Inf: Unpin,
impl<Inf> UnwindSafe for Batched<Inf>where
Inf: UnwindSafe,
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Downcast for Twhere
T: Any,
impl<T> Downcast for Twhere
T: Any,
Source§fn into_any(self: Box<T>) -> Box<dyn Any>
fn into_any(self: Box<T>) -> Box<dyn Any>
Convert
Box<dyn Trait> (where Trait: Downcast) to Box<dyn Any>. Box<dyn Any> can
then be further downcast into Box<ConcreteType> where ConcreteType implements Trait.Source§fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
Convert
Rc<Trait> (where Trait: Downcast) to Rc<Any>. Rc<Any> can then be
further downcast into Rc<ConcreteType> where ConcreteType implements Trait.Source§fn as_any(&self) -> &(dyn Any + 'static)
fn as_any(&self) -> &(dyn Any + 'static)
Convert
&Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &Any’s vtable from &Trait’s.Source§fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
Convert
&mut Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &mut Any’s vtable from &mut Trait’s.Source§impl<T> DowncastSync for T
impl<T> DowncastSync for T
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more