Expand description
Multi-label toxicity scoring for [badwords], on ONNX Runtime.
The rule-based engine in badwords-core answers “is this word in the
dictionary”. This crate answers “how toxic is this text, and in what way”,
with one probability per axis - toxicity, insult, threat, and so on, taken
from the model’s own config rather than hardcoded.
use badwords_ml::ToxicityModel;
let model = ToxicityModel::open("model")?;
let scores = model.predict("you are an idiot")?;
println!("{:.2}", scores.toxicity());
for (axis, value) in scores.above(0.5) {
println!("{axis}: {value:.2}");
}HybridFilter puts the rules in front: they answer outright when they
can, and the model is asked only when they cannot.
The model directory is the one ml/train.py writes - model.onnx,
config.json and tokenizer.json - the same directory the Python package
downloads and caches.
Structs§
- Hybrid
Filter - A rule filter and a model, used together.
- Hybrid
Result - What the hybrid decided, and on what evidence.
- Scores
- Per-axis probabilities for one text.
- Toxicity
Model - A loaded toxicity model.
- Toxicity
Model Builder - Options for loading a model.
Enums§
- Decision
- Which half of the filter produced the verdict.
- Error
- Anything that can go wrong loading or running the model.
Constants§
- DEFAULT_
CERTAIN_ AT - Rule score at or above which the rules answer alone.
1.0is an exact dictionary hit; lower values let a fuzzy near-match decide too. - DEFAULT_
DECISION_ THRESHOLD - Model probability at or above which text counts as toxic.
- DEFAULT_
MAX_ LENGTH - Longest input in tokens. Matches the Python side and the training run.
- REQUIRED_
FILES - Files a usable model directory must contain.
- TOXICITY
- The axis every model annotates, and the one a single number should come from.
Functions§
- cache_
dir - The cache directory the Python package downloads into.
- is_
complete - Whether a directory holds every file the model needs.
- locate_
model - Locate a model:
BADWORDS_ML_PATHfirst, then the shared cache.