Crate hebrew_unicode_utils

Expand description

§Hebrew_Unicode_Utils

Crates.io License Crates.io Version docs.rs

§Project Status

This project is currently in the development mode.

Current Version:
The latest version is v0.4.3.

Stability:
The crate is currently in development mode and has not undergone thorough testing, making it unsuitable for production use at this time. However, it is expected to evolve in the future.

Despite its experimental nature, all released versions available on GitHub are guaranteed to pass the following Cargo sub-commands without any issues:

cargo fmt
cargo check
cargo clippy
cargo build
cargo doc
cargo test

Updates:
While the project is not actively seeking new features or major enhancements, critical bug fixes and security updates will be addressed as needed. Users are encouraged to report any issues they encounter.

§Description

This crate (hebrew_unicode_utils) is a library written in Rust and is built on top of the low-level crate hebrew_unicode_script. This crate will focus on the Unicode Hebrew Block and the Hebrew accent systems.

The functionality of this library can be broadly categorized into the following types:

Removing

This is about removing a certain set of Hebrew character types from a string.
Showing

This category is all about showing a particular type of Hebrew character.

The idea behind this is that this could help people (who want to learn Hebrew) to distinguish the different characters.

Note:
Consonants will always be shown in combination with e.g. vowel characters, otherwise the readability will decrease. For example, if there are multiple vowels in one sentence, then if there are no consonants shown, then all vowels will be displayed on top of each other. Which would make the sentence unreadable.
Statistics

This category contains functionality that gives the user information about the particular statistics of a text string.

For example an answer on the following question: “What Hebrew character types are in my text string?”
Meteg Layout

There are some characters that are used inside the Hebrew text for correct rendering in documents and websites when displaying text.

This category contains the functionality to determine if a character is a meteg layout character or not.

Supported characters are: ZWNJ (zero width non-joiner), ZWJ (zero width joiner) and CGJ (combining grapheme joiner)
Accents

This category facilitates the identification and validation of Hebrew accents as used in the Tanach. Both a check on individual accents and membership of collections are possible.

§Accent system in the Tanach

based on genre
disjunt versus conju
hiarchy groups
sometine consists of two unicode code-points
some accents

§Examples

§Removing characters

use hebrew_unicode_utils::remove_hbr_ligature_yiddish;
    
let test_str = "XװױײZ";
let test_str_filtered = remove_hbr_ligature_yiddish(test_str);

assert_eq!(test_str_filtered.as_ref(),"XZ");

use hebrew_unicode_utils::remove_hbr_accent;

let test_str = "בְּרֵאשִׁ֖ית";
let test_str_filtered = remove_hbr_accent(test_str);

assert_eq!(test_str_filtered.as_ref(), "בְּרֵאשִׁית");

§Showing characters

use hebrew_unicode_utils::show_hbr_mark;

let input_str = "Q מִצְרָ֑יְמָה ה֯";
let input_str_showed = show_hbr_mark(input_str);
        
assert_eq!(input_str_showed.as_ref(), "Q מצרימה ה֯");

use hebrew_unicode_utils::show_hbr_point_semi_vowel;

let input_str = "ֲדְ נָפֶשׁ גֱכֳע";
let input_str_showed = show_hbr_point_semi_vowel(input_str);

assert_eq!(input_str_showed.as_ref(), "ֲדְ נפש גֱכֳע");

§Statistics

use hebrew_unicode_utils::hebrew_character_frequency;
    
let input_string = "Xבהב";
let freq_map = hebrew_character_frequency(input_string);
assert_eq!(freq_map.get(&"X".to_string()), None);
assert_eq!(freq_map.get(&"ב".to_string()), Some(&2));  
assert_eq!(freq_map.get(&"ה".to_string()), Some(&1));

use hebrew_unicode_utils::hebrew_character_types;
use hebrew_unicode_utils::IsPresent;
    
let input_string = "Xבהב";
let type_struct = hebrew_character_types(input_string);
assert_eq!(type_struct.accent, IsPresent::No);
assert_eq!(type_struct.consonant, IsPresent::Yes);
assert_eq!(type_struct.non_hebrew, IsPresent::Yes);

§TODO Meteg layout characters

use hebrew_unicode_utils::hebrew_character_types;
use hebrew_unicode_utils::IsPresent;
    
let input_string = "Xבהב";
let type_struct = hebrew_character_types(input_string);
assert_eq!(type_struct.accent, IsPresent::No);
assert_eq!(type_struct.consonant, IsPresent::Yes);
assert_eq!(type_struct.non_hebrew, IsPresent::Yes);

§TODO Accents - 21 books

use hebrew_unicode_utils::hebrew_character_types;
use hebrew_unicode_utils::IsPresent;
    
let input_string = "Xבהב";
let type_struct = hebrew_character_types(input_string);
assert_eq!(type_struct.accent, IsPresent::No);
assert_eq!(type_struct.consonant, IsPresent::Yes);
assert_eq!(type_struct.non_hebrew, IsPresent::Yes);

§TODO Accents - 3 books

use hebrew_unicode_utils::hebrew_character_types;
use hebrew_unicode_utils::IsPresent;
    
let input_string = "Xבהב";
let type_struct = hebrew_character_types(input_string);
assert_eq!(type_struct.accent, IsPresent::No);
assert_eq!(type_struct.consonant, IsPresent::Yes);
assert_eq!(type_struct.non_hebrew, IsPresent::Yes);

§Release

For an overview of released versions see releases.

§Install

For installation see the hebrew_unicode_utils page at crates.io.

§Safety

All functions are written in safe Rust.

§Panics

No panics for so far I know of.

§Errors

All functions return either a Cow, a Struct or a HashMap.

§License

The hebrew_unicode_utils library is distributed under either of

Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

§Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

§References

Unicode Block Hebrew - chapter 9.1 (meteg)
Basics of HEBREW ACCENTS, written by Mark D. Futato, Sr.
Hebrew Cantillation Marks And Their Encoding by Helmut Richter
Gesenius Hebrew Grammar - §15. The Accents.

§Notes

Vowels are sometimes called Hebrew Points
Accents are sometimes called Hebrew Cantilationmarks

This crate has been inspired by niqqud

Structs§

HebrewCharacterTypeCount
HebrewCharacterTypes

Enums§

IsPresent

Traits§

HebrewMetegJoinerCharacters

Functions§

hebrew_character_frequency: Get the frequency hebrew characters of the given string
hebrew_character_type_cnt: Get Hebrew character types for a given string
hebrew_character_types: Get Hebrew character types for a given string
is_combining_grapheme_joiner: Checks if the given character is a: CGJ (combining grapheme joiner)
is_zero_width_joiner: Checks if the given character is a: ZWJ (zero width joiner)
is_zero_width_non_joiner: Checks if the given character is a: ZWNJ (zero width non-joiner)
remove_hbr_accent: Removes all Hebrew acccents from the given string.
remove_hbr_block: Removes all characters belonging to the unicode block ‘Hebrew’ from the given string.
remove_hbr_consonant: Removes all Hebrew letters (final and normal) from the given string.
remove_hbr_consonant_final: Removes all Hebrew final letters from the given string.
remove_hbr_consonant_normal: Removes all Hebrew normal letters from the given string.
remove_hbr_ligature_yiddish: Removes all Yiddish ligatures from the given string.
remove_hbr_mark: Removes all Hebrew marks from the given string.
remove_hbr_point: Removes all Hebrew points from the given string.
remove_hbr_point_reading_sign: Removes all Hebrew point reading signs from the given string.
remove_hbr_point_semi_vowel: Removes all Hebrew point semi-vowels from the given string.
remove_hbr_point_vowel: Removes all Hebrew point vowels from the given string.
remove_hbr_punctuation: Removes all Hebrew punctuations from the given string.
remove_hbr_yod_triangle: Removes all Hebrew yod triangles from the given string.
show_hbr_accent: Shows all Hebrew acccents that are found in the given string.
show_hbr_consonant: Shows all Hebrew letters (final and normal) that are found in the given string.
show_hbr_consonant_final: Shows all Hebrew final letters that are found in the given string.
show_hbr_consonant_normal: Shows all Hebrew normal letters that are found in the given string.
show_hbr_ligature_yiddish: Shows all Yiddish ligatures that are found in the given string.
show_hbr_mark: Shows all Hebrew marks that are found in the given string.
show_hbr_point: Shows all Hebrew points that are found in the given string.
show_hbr_point_reading_sign: Shows all Hebrew point reading signs that are found in the given string.
show_hbr_point_semi_vowel: Shows all Hebrew point semi-vowels that are found in the given string.
show_hbr_point_vowel: Shows all Hebrew point vowels that are found in the given string.
show_hbr_punctuation: Shows all Hebrew punctuations that are found in the given string.
show_hbr_yod_triangle: Shows all Hebrew yod triangles that are found in the given string.