pub unsafe extern "C" fn llama_sample_classifier_free_guidance(
ctx: *mut llama_context,
candidates: *mut llama_token_data_array,
guidance_ctx: *mut llama_context,
scale: f32
)Expand description
@details Apply classifier-free guidance to the logits as described in academic paper “Stay on topic with Classifier-Free Guidance” https://arxiv.org/abs/2306.17806
@param candidates A vector of llama_token_data containing the candidate tokens, the logits must be directly extracted from the original generation context without being sorted.
@params guidance_ctx A separate context from the same model. Other than a negative prompt at the beginning, it should have all generated and user input tokens copied from the main context.
@params scale Guidance strength. 1.0f means no guidance. Higher values mean stronger guidance.