pub struct InferenceRequest<'a> {
pub prompt: &'a str,
pub parameters: Option<&'a InferenceParameters>,
pub play_back_previous_tokens: bool,
pub maximum_token_count: Option<usize>,
}
Expand description
Settings specific to InferenceSession::infer.
Fields§
§prompt: &'a str
The prompt to feed to the model.
parameters: Option<&'a InferenceParameters>
The parameters to use during this inference attempt. If not specified, this will default to the parameters specified in the model.
play_back_previous_tokens: bool
Whether or not to call the callback with the previous tokens that were encountered in this session.
You likely want to turn this on if you’re using a session that has been rehydrated from a snapshot.
maximum_token_count: Option<usize>
The maximum number of tokens to generate.
Trait Implementations§
Source§impl<'a> Clone for InferenceRequest<'a>
impl<'a> Clone for InferenceRequest<'a>
Source§fn clone(&self) -> InferenceRequest<'a>
fn clone(&self) -> InferenceRequest<'a>
Returns a duplicate of the value. Read more
1.0.0 · Source§const fn clone_from(&mut self, source: &Self)
const fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moreSource§impl<'a> Debug for InferenceRequest<'a>
impl<'a> Debug for InferenceRequest<'a>
Source§impl<'a> Default for InferenceRequest<'a>
impl<'a> Default for InferenceRequest<'a>
Source§fn default() -> InferenceRequest<'a>
fn default() -> InferenceRequest<'a>
Returns the “default value” for a type. Read more
Source§impl<'a> PartialEq for InferenceRequest<'a>
impl<'a> PartialEq for InferenceRequest<'a>
impl<'a> Copy for InferenceRequest<'a>
impl<'a> StructuralPartialEq for InferenceRequest<'a>
Auto Trait Implementations§
impl<'a> Freeze for InferenceRequest<'a>
impl<'a> RefUnwindSafe for InferenceRequest<'a>
impl<'a> Send for InferenceRequest<'a>
impl<'a> Sync for InferenceRequest<'a>
impl<'a> Unpin for InferenceRequest<'a>
impl<'a> UnwindSafe for InferenceRequest<'a>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more