nixel 5.2.0

Features

• ✔️ Fast

  It parses all the files in Nixpkgs in under 25 seconds, single-threaded. [^benchmark-specs]

  It's written in Rust and a little bit of C++, Flex and GNU Bison.

• ✔️ Correct

  This library is a copy-paste of the original lexer and parser of Nix, with some types adapted for better ergonomy.

  No parser can get closer to the original implementation than this.

• ✔️ Reliable

  High coverage, battle-tested, and memory-safe[^memory-safe].

• ✔️ Useful

  It gives you comments, whitespace, starting and end positions, automatic string un-escaping, multiline string indentation handling, a typed API, and everything you need to parse the Nix language!

Usage

You can check out the documentation at docs.rs/nixel.

This is a full usage example:

let input: String = String::from(
    r#"
        # Greet the user
        "Hello, World!"
        # Bye!
    "#,
);

let parsed: nixel::Parsed = nixel::parse(input);

match &*parsed.expression {
    nixel::Expression::String(string) => {
        assert_eq!(
            &string.span,
            &nixel::Span {
                start: nixel::Position { line: 3, column: 9 }.into(),
                end: nixel::Position { line: 3, column: 24 }.into(),
            }
            .into()
        );
        assert_eq!(
            &parsed.trivia_before(&string.span.start)[1],
            &nixel::Trivia::Comment(nixel::TriviaComment {
                content: "# Greet the user".into(),
                span: nixel::Span {
                    start: nixel::Position { line: 2, column: 9 }.into(),
                    end: nixel::Position { line: 2, column: 25 }.into(),
                }
                .into()
            })
        );
        assert_eq!(
            &string.parts[0],
            &nixel::Part::Raw(nixel::PartRaw {
                content: "Hello, World!".into(),
                span: nixel::Span {
                    start: nixel::Position { line: 3, column: 10 }.into(),
                    end: nixel::Position { line: 3, column: 23 }.into(),
                }
                .into()
            })
        );
        assert_eq!(
            &parsed.trivia_after(&string.span.end)[1],
            &nixel::Trivia::Comment(nixel::TriviaComment {
                content: "# Bye!".into(),
                span: nixel::Span {
                    start: nixel::Position { line: 4, column: 9 }.into(),
                    end: nixel::Position { line: 4, column: 15 }.into(),
                }
                .into()
            })
        );
    },
    expression => unreachable!("Expected a String, got: {expression:#?}"),
}

Or from the CLI using Rust's Debug trait:

$ echo '1 + 2' | nix run github:kamadorueda/nixel -- --format=debug

BinaryOperation(
    BinaryOperation {
        left: Integer(
            Integer {
                value: "1",
                span: Span {
                    start: Position {
                        line: 1,
                        column: 1,
                    },
                    end: Position {
                        line: 1,
                        column: 2,
                    },
                },
            },
        ),
        operator: Addition,
        right: Integer(
            Integer {
                value: "2",
                span: Span {
                    start: Position {
                        line: 1,
                        column: 5,
                    },
                    end: Position {
                        line: 1,
                        column: 6,
                    },
                },
            },
        ),
    },
)

Or from the CLI using JSON format:

$ echo '1 + 2' | nix run github:kamadorueda/nixel -- --format=json

{
  "BinaryOperation": {
    "left": {
      "Integer": {
        "value": "1",
        "span": {
          "start": {
            "line": 1,
            "column": 1
          },
          "end": {
            "line": 1,
            "column": 2
          }
        }
      }
    },
    "operator": "Addition",
    "right": {
      "Integer": {
        "value": "2",
        "span": {
          "start": {
            "line": 1,
            "column": 5
          },
          "end": {
            "line": 1,
            "column": 6
          }
        }
      }
    }
  }
}

You can check out more examples in the tests folder.

Alternatives

• rnix-parser.

License

Please read LICENSE.md.

Footnotes

[^benchmark-specs]: Running on a machine with:

- CPU: 4 physical, 4 logical, 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
- MHz: from 400 to 4700 MHz
- BogoMips: 5606.40
- Cache L3: 12 MiB

The following command takes around 1 minute:

```bash
$ nix build --system x86_64-linux
$ time find /data/nixpkgs -type f -name '*.nix' \
  -exec ./result/bin/nixel --format=none {} \;

real  0m24.293s
user  0m15.066s
sys   0m8.955s
```

[^memory-safe]: Tested under real-life workloads using Valgrind, and by running an infinite loop of parsing cycles over Nixpkgs :).

```bash
$ nix build --system x86_64-linux
$ valgrind ./result/bin/nixel $file

  LEAK SUMMARY:
    definitely lost: 0 bytes in 0 blocks
    indirectly lost: 0 bytes in 0 blocks
      possibly lost: 0 bytes in 0 blocks
         suppressed: 0 bytes in 0 blocks
```