Type Alias kalosm_language_model::BoxedSyncModel
source · pub type BoxedSyncModel = Box<dyn SyncModel<Session = AnySession>>;Expand description
A trait object for a sync model.
Aliased Type§
struct BoxedSyncModel(/* private fields */);Trait Implementations§
source§impl SyncModel for BoxedSyncModel
impl SyncModel for BoxedSyncModel
§type Session = AnySession
type Session = AnySession
The session type for this model.
source§fn new_session(&self) -> Result<Self::Session>
fn new_session(&self) -> Result<Self::Session>
Create a new session for this model.
source§fn feed_text(
&self,
session: &mut Self::Session,
prompt: &str,
top_k: Option<usize>
) -> Result<Logits>
fn feed_text( &self, session: &mut Self::Session, prompt: &str, top_k: Option<usize> ) -> Result<Logits>
Run the model synchronously. The model implementation may choose to return only the top k logits.
source§fn feed_tokens(
&self,
session: &mut Self::Session,
tokens: &[u32],
top_k: Option<usize>
) -> Result<Logits>
fn feed_tokens( &self, session: &mut Self::Session, tokens: &[u32], top_k: Option<usize> ) -> Result<Logits>
Run the model synchronously with a pre-tokenized input. The model implementation may choose to return only the top k logits.
source§fn stop_token(&self) -> Result<u32>
fn stop_token(&self) -> Result<u32>
Get the token ID that represents the end of a sequence.