pub fn csv_string_matcher(
quote: impl Into<String>,
) -> impl Fn(&Options) -> Option<ImperativeLexMatcher> + Send + Sync + 'staticExpand description
The custom CSV string matcher factory: "a""b" is the text a"b.
A reduced copy of the standard jsonic string matcher, with the doubled
quote as the only escape. It is the counterpart of the TypeScript
buildCsvStringMatcher export and the Go BuildCsvStringMatcher, in
the shape the engine’s options.lex.match.<name>.make reference takes:
register it with Tabnas::lex_match_factory_ref and name it from an
options document.
use tabnas::GrammarSpec;
let mut parser = tabnas_jsonic::make();
parser.lex_match_factory_ref("@csv-string", tabnas_csv::csv_string_matcher("\""));
let spec = GrammarSpec::from_value(serde_json::json!({
"options": { "lex": { "match": {
"stringcsv": { "order": 100000, "make": "@csv-string" },
} } },
}))?;
parser.grammar(&spec)?;
assert_eq!(parser.parse(r#""a""b""#)?, tabnas::Value::String("a\"b".to_string()));The matcher runs only where the engine wants a string token, and only
at the quote character. A line character inside the quotes is field
text (a quoted field may span lines). Any other control character is
field text too when the engine’s string.allowControl option is on,
as the CSV plugin sets it, and unprintable when it is off, as it is
by default on a plain instance. A quote left open at the end of the
source is unterminated_string. Loop exhaustion is what detects the
open quote, so an odd number of quotes (""", """"") is caught
rather than read as a terminated string.
A quote that is not exactly one UTF-16 code unit leaves the matcher
inert, and the factory answers None rather than installing one. The
canonical opens a quoted field with
quoteMap = { [options.string.quote]: true } and tests
quoteMap[src[sI]], where src[sI] is ONE UTF-16 code unit, so it
can only ever fire for a quote that is one code unit: the empty
string, a multi-character quote such as ab, and an astral character
such as U+1F600, which JavaScript holds as a surrogate PAIR, are all
inert there. Go carries the same guard, with the same reasoning.