pub struct TextGenerator<'a> { /* private fields */ }Expand description
Text generator for autoregressive text generation.
Implementations§
Source§impl<'a> TextGenerator<'a>
impl<'a> TextGenerator<'a>
pub fn new(model: &'a Transformer, device: &'a Device) -> Self
Sourcepub fn generate(
&self,
prompt_tokens: &[u32],
max_new_tokens: usize,
temperature: f64,
top_k: Option<usize>,
) -> Result<Vec<u32>>
pub fn generate( &self, prompt_tokens: &[u32], max_new_tokens: usize, temperature: f64, top_k: Option<usize>, ) -> Result<Vec<u32>>
Generate tokens autoregressively from a prompt.
§Arguments
prompt_tokens- Initial token sequencemax_new_tokens- Maximum number of new tokens to generatetemperature- Sampling temperature (1.0 = no scaling)top_k- Optional top-k filtering
Auto Trait Implementations§
impl<'a> Freeze for TextGenerator<'a>
impl<'a> !RefUnwindSafe for TextGenerator<'a>
impl<'a> Send for TextGenerator<'a>
impl<'a> Sync for TextGenerator<'a>
impl<'a> Unpin for TextGenerator<'a>
impl<'a> UnsafeUnpin for TextGenerator<'a>
impl<'a> !UnwindSafe for TextGenerator<'a>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more