Module llm_base::model::common

Common functions for model evaluation

Functions

• extract_embeddings
  Extract embeddings from EvaluateOutputRequest evaluation
• extract_logits
  Extract logits from EvaluateOutputRequest evaluation
• prepare_for_evaluate
  Common code to prepare a model to evaluate input
• read_last_token
  Return result for just the last token
• update_session
  Update an InferenceSession after evaluation