Struct OnnxCrossEncoder 

pub struct OnnxCrossEncoder { /* private fields */ }

Local int8 cross-encoder reranker on ONNX Runtime.

Implementations

impl OnnxCrossEncoder

pub async fn load(model_id: &str, max_length: usize) -> Result<Self>

Download (if needed) the int8 export for this CPU and the tokenizer from the HuggingFace Hub. Each pair is truncated to max_length tokens, document side first.

pub fn from_files( weights: &Path, tokenizer: &Path, max_length: usize, ) -> Result<Self>

Load from a local .onnx graph and tokenizer.json.

Trait Implementations

impl Reranker for OnnxCrossEncoder

fn score<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, query: &'life1 str, documents: &'life2 [String], ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

One relevance in [0, 1] per document, in input order; higher is more relevant.