= i18n_lexer Rizzen Yazston :url-unicode: https://home.unicode.org/ :CLDR: https://cldr.unicode.org/ :icu4x: https://github.com/unicode-org/icu4x
String lexer and resultant tokens.
TThe Lexer
is initialised using DataProvider
enum of supported data providers to an {url-unicode}[Unicode Consortium] {CLDR}[CLDR] data repository (even a custom database). Usually the repository is just a local copy of the CLDR in the application's data directory. Once the Lexer
has been initialised using new()
method, it may be used to tokenise strings, without needing to re-initialising the Lexer
before use.
Consult the {icu4x}[ICU4X] website for instructions on generating a suitable data repository for the application, by leaving out data that is not used by the application.
Strings are tokenised using the method tokenise()
taking string slice and a vector containing grammar syntax characters.
== Features
Available features for i18n_icu
crate:
-
compiled_data
(Preferred): Enable thecompiled_data
feature oni18n_icu
. Allow for the internal data of the various ICU4X components. -
blob
: Enable theblob
feature oni18n_icu
. Allow for instances ofBlobDataProvider
to be used various ICU4X components that supports {BufferProvider}[BufferProvider
]. -
fs
: Enable thefs
feature oni18n_icu
. Allow for instances ofFsDataProvider
to be used various ICU4X components that supportsBufferProvider
. -
sync
: Allow for rust's concurrency capabilities to be used. Use ofArc
andMutex
insteadRc
andRefCell
. -
log
: Enables logging oni18n_icu
crate.
== Acknowledgement
Stefano Angeleri for advice on various design aspects of implementing the components of the internationalisation project, and also providing the Italian translation of error message strings.