Crate evfmt 

evfmt is both a command-line formatter and a Rust library for normalizing text/emoji variation selectors.

Most callers will want format_text together with Policy.

Stability

This library API is experimental. evfmt follows Cargo’s SemVer compatibility conventions.

Examples

Use format_text for whole-input canonicalization under one Policy. In the example below, #\u{FE0E} is NUMBER SIGN followed by VS15, and \u{00A9} is a bare COPYRIGHT SIGN. Under the default policy, #\u{FE0E} loses the redundant variation selector, while bare \u{00A9} is canonicalized to \u{00A9}\u{FE0E} because it is text-default.

use evfmt::{FormatResult, Policy, format_text};

let policy = Policy::default();

assert_eq!(
    format_text("#\u{FE0E}", &policy),
    FormatResult::Changed("#".to_owned())
);
assert_eq!(
    format_text("\u{00A9}", &policy),
    FormatResult::Changed("\u{00A9}\u{FE0E}".to_owned())
);
assert_eq!(format_text("\u{2728}", &policy), FormatResult::Unchanged);

For interactive repair or editor integrations, scan the input and then work item-by-item. In the next example, A\u{FE0F} contains an unsanctioned presentation selector after A, and the caller chooses to apply the formatter’s fixed canonical replacement. For the built-in evfmt decisions, callers can build repaired output from the original scanned items without rescanning after each selected decision. Walk the original items in order, keeping item.raw for unchanged items and substituting the selected canonical replacement for findings.

use evfmt::{Policy, ScanKind, scan};
use evfmt::analysis::analyze_scan_item;

let policy = Policy::default();
let input = "A\u{FE0F}";

let mut items = scan(input);
assert!(matches!(items.next().unwrap().kind, ScanKind::Passthrough));
let item = items.next().unwrap();
assert!(matches!(
    item.kind,
    ScanKind::UnsanctionedPresentationSelectors(_)
));

let finding = analyze_scan_item(&item, &policy).unwrap();
let non_canonicality = finding.non_canonicality();
assert!(!non_canonicality.is_empty());
assert_eq!(non_canonicality.unsanctioned_selectors, 1);
assert_eq!(finding.default_decisions().len(), 0);

let repaired = finding.canonical_replacement_with_decisions(&[]).unwrap();
assert_eq!(repaired, "");

The analysis API is the usual entry point for interactive fixing. It analyzes scanned items under the supplied Policy and returns the whole-item canonical replacement available for each non-canonical finding. Ambiguous selector slots are represented as source-order presentation decisions; fixed repairs have no decisions and use the empty decision vector.

Custom policies can be built from policy_key_set policy key sets. In this example, a singleton policy key for \u{00A9} allows bare COPYRIGHT SIGN to remain bare.

use evfmt::{Policy, PolicyKeySet, format_text, policy_key_set};

let ascii_and_copyright = policy_key_set::ASCII | PolicyKeySet::singleton('\u{00A9}');
let policy = Policy::default().with_prefer_bare(ascii_and_copyright);

let formatted = format_text("\u{00A9}", &policy);
assert_eq!(formatted, evfmt::FormatResult::Unchanged);

Here “variation-sequence character” means a character listed in Unicode’s emoji-variation-sequences.txt.

Public module boundaries:

• the crate root is the high-level API: whole-input formatting and convenience analysis helpers
• policy defines formatter policy configuration
• formatter owns whole-text formatting
• analysis analyzes scanned items under policy and reports non-canonicality plus available replacements
• presentation defines the text/emoji presentation decision shared by scanning and analysis
• scanner owns structural tokenization into singletons, keycaps, ZWJ chains, standalone variation selector runs, and passthrough slices
• policy_key_set defines the typed PolicyKeySet model used by the library policy API

Re-exports

pub use analysis::Finding;

pub use analysis::NonCanonicality;

pub use analysis::analyze_scan_item;

pub use formatter::FormatResult;

pub use formatter::format_text;

pub use policy::Policy;

pub use policy_key_set::PolicyKeySet;

pub use presentation::Presentation;

pub use scanner::ScanItem;

pub use scanner::ScanKind;

pub use scanner::Scanner;

pub use scanner::scan;

Modules

analysis

Policy-aware analysis for scanned emoji variation structures.

formatter

Core formatting engine.

policy

Formatter policy configuration.

policy_key_set

Finite policy-key sets for formatter policy.

presentation

Presentation style requested by text/emoji variation selectors.

scanner

Sequence-aware scanner for text/emoji variation sequences.