serde implementation of deb822 format

Deb822 format is used by Debian and derivatives distros in package (source and binary), repositories and sources.list/*.sources. We would implement all thing needed to parse theses files.

API

# use std::collections::BTreeMap;
# use std::io::Cursor;

let stanza = "A: B";
let file = "A: B\n\nC: D\n\n";
let expect_stanza = BTreeMap::from([("A", "B")]);
let expect_stanza_buf = BTreeMap::from([("A".to_string(), "B".to_string())]);
let expect_file = vec![BTreeMap::from([("A", "B")]),BTreeMap::from([("C", "D")])];
let expect_file_buf = vec![BTreeMap::from([("A".to_string(), "B".to_string())]),BTreeMap::from([("C".to_string(), "D".to_string())])];

assert_eq!(expect_stanza, deb822::stanza::from_str(stanza).unwrap());
assert_eq!(expect_stanza, deb822::stanza::from_slice(stanza.as_bytes()).unwrap());
assert_eq!(expect_stanza_buf, deb822::stanza::from_buf_reader(Cursor::new(stanza)).unwrap());

assert_eq!(expect_file, deb822::from_str::<Vec<_>>(file).unwrap());
assert_eq!(expect_file, deb822::from_slice::<Vec<_>>(file.as_bytes()).unwrap());
assert_eq!(expect_file_buf, deb822::from_buf_reader::<Vec<_>, _>(Cursor::new(file)).unwrap());

assert_eq!(stanza, deb822::to_string(&expect_stanza).unwrap());
assert_eq!(stanza.as_bytes(), deb822::to_vec(&expect_stanza).unwrap());
let mut output = vec![];
deb822::to_writer(Cursor::new(&mut output), &expect_stanza).unwrap();
assert_eq!(stanza.as_bytes(), output);

assert_eq!(file, deb822::to_string(&expect_file).unwrap());
assert_eq!(file.as_bytes(), deb822::to_vec(&expect_file).unwrap());
let mut output = vec![];
deb822::to_writer(Cursor::new(&mut output), &expect_file).unwrap();
assert_eq!(file.as_bytes(), output);

The format

Deb822 format is documentation can be found in deb822(5).

Comment

Line start by # is a comment-line and can be present anywhere in file, including in the middle of a multiline value.

Field name

Field name must be only contains ASCII char, excluding control characters, space and : and not begins with # or -.

Field name and value are separated by a :.

[!NOTE] Some invalid key can be parsed in some case. Restriction applied is not start with space or # and not contains any dash # or newline \n

[!INFO] Deb822 documentation tell field name are case insensitive, this isn't implemented, we need to use as key a str-insensitive type

# use std::collections::BTreeMap;

let s = "-Invalid: Value1
Invali\rd: Value2
Invalid key: Value3
Invalidéééé: Value4
Ke#1: Value5
Valid-Key: Value6";

let map: BTreeMap<&str, &str> = deb822::from_str(s).unwrap();

assert_eq!(map["-Invalid"], "Value1");
assert_eq!(map["Invali\rd"], "Value2");
assert_eq!(map["Invalid key"], "Value3");
assert_eq!(map["Invalidéééé"], "Value4");
assert_eq!(map["Ke#1"], "Value5");
assert_eq!(map["Valid-Key"], "Value6");

assert_eq!(deb822::to_string(&map).unwrap(), s);

Value

Simple

Simple value are a all following colon (:) until the end of the line. One space after colon are trimmed.

# use std::collections::BTreeMap;

let s = r#"Key1: Value1
Key2:Value2
Key3:  Value3
Key4:  Value4  "#;

let map: BTreeMap<&str, &str> = deb822::from_str(s).unwrap();

assert_eq!(map["Key1"], "Value1");
assert_eq!(map["Key2"], "Value2");
assert_eq!(map["Key3"], " Value3");
assert_eq!(map["Key4"], " Value4  ");

let equivalent_fmt = r#"Key1: Value1
Key2: Value2
Key3:  Value3
Key4:  Value4  "#;

assert_eq!(deb822::to_string(&map).unwrap(), equivalent_fmt);

Folded

A folded value is simply a value continued to next line. Continuation line are prefixed by a space .

Deserialize into a String just work, but for borrow-deserialize (e.g. into &str) isn't possible with multi-line value (a space removed at begin of each line). Borrow-deserialize each line separated is possible (e.g. into Vec<&str>).

# use std::collections::BTreeMap;

let s = r#"Key1: Value1
 On multiple line
Key2: Value2
 On multiple line
 .
 With empty line in middle
"#;

let map: BTreeMap<&str, Vec<&str>> = deb822::from_str(s).unwrap();

assert_eq!(map["Key1"], ["Value1", "On multiple line"]);
assert_eq!(map["Key2"], ["Value2", "On multiple line", "", "With empty line in middle"]);

dbg!(&map);
println!("{s}");
println!("{}", deb822::to_string(&map).unwrap());
assert_eq!(deb822::to_string(&map).unwrap(), s);


let map: BTreeMap<&str, String> = deb822::from_str(s).unwrap();

assert_eq!(map["Key1"], "Value1\nOn multiple line");
assert_eq!(map["Key2"], "Value2\nOn multiple line\n\nWith empty line in middle");

assert_eq!(deb822::to_string(&map).unwrap(), s.trim());

Multiline

A multiline value has same syntax as folded value, but newline are signifiant. It is equals to folded value (this implementation keep always newline).

In some file first line (after colon) are different means as continuations lines. For that you can deserialize it into a pair.

# use std::collections::BTreeMap;

# // NOTE: comment in markdown must start with double '#' because first is
# // removed by cargo doc for hide line

let s = r#"
Key1: Value1
 On multiple line
Key2: Value2
 On multiple line
## Comment are allowed inside mutli-line value
 .
 With empty line in middle
"#;

let comment_trimmed = r#"Key1: Value1
 On multiple line
Key2: Value2
 On multiple line
 .
 With empty line in middle
"#;

let map: BTreeMap<&str, (&str, Vec<&str>)> = deb822::from_str(s).unwrap();

assert_eq!(map["Key1"], ("Value1", vec!["On multiple line"]));
assert_eq!(
  map["Key2"],
  ("Value2", vec!["On multiple line", "", "With empty line in middle"])
);

assert_eq!(deb822::to_string(&map).unwrap(), comment_trimmed);

let map: BTreeMap<&str, (&str, String)> = deb822::from_str(s).unwrap();

assert_eq!(map["Key1"], ("Value1", "On multiple line".to_string()));
assert_eq!(map["Key2"], ("Value2", "On multiple line\n\nWith empty line in middle".to_string()));

assert_eq!(deb822::to_string(&map).unwrap(), comment_trimmed);

bool on deb822

• true are serialize to "yes"
• false are serialize to "no"

For now only yes and no value are recognized as boolean, but in future some other string can be added to list.

integer and float on deb822

Both are deserilized by using FromStr trait and serialized with Display trait.

Stanza

Stanza is just a list of key-value line.

Document

A document is one or more stanzas of fields. Stanzas are separated by empty line (really empty or only contains space and tab).

# use std::collections::HashMap;

let s = r#"
Stanza1: Value1

Stanza2: Value2
  
Stanza3: Value2
"#;

let expected = r#"Stanza1: Value1

Stanza2: Value2

Stanza3: Value2

"#;

let list: Vec<HashMap<&str, &str>> = deb822::from_str(s).unwrap();

assert_eq!(list.len(), 3);

assert_eq!(deb822::to_string(&list).unwrap(), expected);

# use std::collections::HashMap;

let s = r#"
Stanza1: Value1
## a Comment

## a Comment

## a Comment
Stanza2: Value2
"#;

let expected = r#"Stanza1: Value1

Stanza2: Value2

"#;

let list: Vec<HashMap<&str, &str>> = deb822::from_str(s).unwrap();

assert_eq!(list.len(), 2);

assert_eq!(deb822::to_string(&list).unwrap(), expected);

The first stanza of a document can be have special means, you can deserialize the document to a pair for extract it automatically.

# use std::collections::HashMap;

let s = r#"Stanza1: Value1

Stanza2: Value2

Stanza3: Value3

"#;

type Stanza<'a> = HashMap<&'a str, &'a str>;
let list: (Stanza, Vec<Stanza>) = deb822::from_str(s).unwrap();

assert_eq!(list.1.len(), 2);

assert_eq!(deb822::to_string(&list).unwrap(), s);

Use with custom struct

Implement serializer and deserializer for serde.

# use serde_derive::{Deserialize, Serialize};
#[derive(Serialize, Deserialize)]
// In deb822, we not have standard, some field named on PascalCase and other
// with title-capitalized kebab-case (not supported by serde)
#[serde(rename_all = "PascalCase")]
struct Info<'a> {
  // Extract value with zero-copy
  #[serde(rename="Borrow-Str")]
  borrow_str: &'a str,
  // or by copy
  string: String,

  // 'yes' and 'no' value are parsed as bool
  flag: bool,

  // number (int and float) are parsed with [`FromStr::from_str`]
  size: u64,

  // When value is 2-tuple, first element is the first line and second element
  // is followed line (The second value can't be borrowed, because one space
  // at start line must be removed)
  description: (&'a str, String),
  // You can also extract each line separated
  description_alt: (&'a str, Vec<&'a str>),
  // or not extract separate first-line
  description_alt2:  Vec<&'a str>,
}

let s = r#"
Borrow-Str: string one
#Comment are removed
#first space after colon (:) are removed if present
String: string two
Flag: yes
Size: 25677
Description: title line
 continuation line
 on two line
DescriptionAlt: title line
 continuation line
 on two line
DescriptionAlt2: title line
 continuation line
 on two line
"#;

let _: Info = deb822::from_str(s).unwrap();