llama_cpp_2::context

Module sample

Source
Expand description

Sampling functions for the context.

Modulesยง

  • Create a sampler struct to encapsulate the sampling process. This allows passing all the possible sampling parameters around as a single struct, and also allow late binding of expensive context like crate::context::LlamaContext or token history to the sampler.