Skip to main content

strip_field_padding

Function strip_field_padding 

Source
pub fn strip_field_padding(input: &[u8], delimiter: u8) -> Cow<'_, [u8]>
Expand description

Delete the whitespace padding that “pretty printed” CSVs put around quoted fields, so the parser can see they were quoted at all.

RFC 4180 only recognises a quote that is the first byte of a field, so a human-aligned file like

"Index", "Name", "Day"
 1, "George Washington",  22

parses the second column’s header as the literal text "Name" — padding, quotes and all — which is why SELECT Name then fails with “column not found”. Worse, a comma inside a padded field splits it in two, because the parser never saw the field as quoted.

Nobody writing a file like that intends the quotes to end up in their column names, so we delete the run of spaces/tabs in front of an opening quote (and the mirror run after the closing quote). That turns the field back into a properly quoted one and the parser handles it from there — embedded delimiters included.

Deliberately narrow: unquoted fields are left exactly as they are. In a file written by a machine, 22 and 22 mean the same thing, but there is no way to tell that apart from a value whose spaces are real — an unquoted David is the only way some producers can express a padded string, and data/test_simple_strings.csv relies on it surviving the load. Whitespace next to a quote carries no such ambiguity: the quotes already delimit the value, so anything outside them is alignment. Numeric-looking unquoted fields get their padding handled at type-inference time instead, where trimming can’t destroy a string.

Bytes inside a quoted field are copied verbatim, so " padded " keeps its spaces — if you quoted the whitespace, you meant it. The delimiter itself is never treated as padding, which keeps .tsv files intact.

Returns Cow::Borrowed when there was nothing to strip, so the common RFC-4180 case costs one scan and no allocation.