Struct llama_cpp::CompletionHandle
source · pub struct CompletionHandle { /* private fields */ }
Expand description
A handle (and channel) to an ongoing completion job on an off thread.
If this structure is dropped, the off thread is stopped.
Implementations§
source§impl CompletionHandle
impl CompletionHandle
sourcepub fn next_token(&self) -> Option<Token>
pub fn next_token(&self) -> Option<Token>
Blocks the current thread, resolving to the next completed token, or None
if EOS is
reached.
sourcepub async fn next_token_async(&self) -> Option<Token>
pub async fn next_token_async(&self) -> Option<Token>
Asynchronously yields the current thread, resolving to the next completed token, or None
if EOS is reached.
Auto Trait Implementations§
impl RefUnwindSafe for CompletionHandle
impl Send for CompletionHandle
impl Sync for CompletionHandle
impl Unpin for CompletionHandle
impl UnwindSafe for CompletionHandle
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more