Crate gibberish_or_not

Source

Structs§

GibberishDetector
Gibberish detector with optional enhanced model

Enums§

ModelError
Errors that can occur during model operations
Sensitivity
Sensitivity level for gibberish detection
TokenStatus
Status of the HuggingFace token

Functions§

calculate_non_printable_ratio
Calculate the ratio of non-printable characters in a string Returns a value between 0.0 (all printable) and 1.0 (all non-printable)
check_token_status
Check if HuggingFace token is required and available
default_model_path
Get default model path in user’s cache directory
download_model
Download model files with progress reporting
download_model_with_progress_bar
Download model with a simple progress bar
is_gibberish
Checks if the given text is gibberish based on English word presence and n-gram analysis scores. The sensitivity level determines how strict the classification should be.
is_password
Checks if the given text matches a known common password.
is_string_bad_quality
is it worth it to calculate this string?
model_exists
Check if model exists at given path