Skip to main content

Crate drama_llama

Crate drama_llama 

Source

Re-exports§

pub use prompt::Message;
pub use prompt::Prompt;
pub use prompt::Role;

Modules§

cli
data
prompt

Structs§

CandidatePredictor
An iterator that predicts a sequence of tokens until the end of the sequence is reached.
Candidates
A container for candidate tokens.
Engine
An Engine encompasses everything needed to run inferences. It contains the model and the context. It is the main entry point for running inferences.
InvalidProbability
Error for invalid probability values.
Model
An ergonomic wrapper for a llama.cpp model.
NGram
An immutable N-gram of tokens.
NGramData
Metadata about an Ngram.
NGramStats
A map of NGram metadata.
PiecePredictor
A predictor that predicts pieces of text.
PredictOptions
Options for prediction.
Predicted
Contains a token and the associated piece. This is a convenience struct to avoid ackward iterator usage when both the token and piece are needed.
Predictor
Probability
A Probability is a wrapper around a floating point number that represents a probability. It is guaranteed to be between 0.0 and 1.0.
RepetitionOptions
Options for apply_sample_repetition_penalties.
SampleOptions
Options determining how raw logits are turned into a token. This is used by Candidates::sample_token and associated functions.
TokenDataArray
TokenPredictor
Vocab

Enums§

NewError
Possible errors when creating a new Engine.
RepetitionError
SamplingMode
Sorted
Sort state of the candidates.
VocabKind

Constants§

TOS

Functions§

llama_quantize
Quantize a Llama model.