Struct llama_cpp::CompletionHandle
source · pub struct CompletionHandle<'a> { /* private fields */ }
Expand description
A handle (and channel) to an ongoing completion job on an off thread.
If this structure is dropped, the off thread is stopped.
Implementations§
source§impl<'a> CompletionHandle<'a>
impl<'a> CompletionHandle<'a>
sourcepub fn next_token(&mut self) -> Option<CompletionToken<'_>>
pub fn next_token(&mut self) -> Option<CompletionToken<'_>>
Blocks the current thread, resolving to the next completed token, or None
if EOS is
reached.
sourcepub async fn next_token_async(&mut self) -> Option<CompletionToken<'_>>
pub async fn next_token_async(&mut self) -> Option<CompletionToken<'_>>
Asynchronously yields the current thread, resolving to the next completed token, or None
if EOS is reached.
Auto Trait Implementations§
impl<'a> !RefUnwindSafe for CompletionHandle<'a>
impl<'a> Send for CompletionHandle<'a>
impl<'a> Sync for CompletionHandle<'a>
impl<'a> Unpin for CompletionHandle<'a>
impl<'a> !UnwindSafe for CompletionHandle<'a>
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more