Macro unstringify::unstringify

unstringify!() { /* proc-macro */ }

Reverse of stringify!: tokenize an input string literal.

Basic example

use ::unstringify::unstringify;

unstringify!(r#"
    fn main ()
    {
        println!("Hello, World!");
    }
"#);

Or, equivalently:

use ::unstringify::unstringify;

unstringify!(stringify! {
    fn main ()
    {
        println!("Hello, World!");
    }
});

A more interesting example

A (non-procedural!) macro to evaluate the rust code snippets inside docstrings:

use ::unstringify::unstringify;

macro_rules! eval_docstrings {(
    $(
        #[doc = $doc:tt]
    )*
) => (
    extract_code_snippets! {
        @find_start
        $($doc)*
    }
)}

macro_rules! extract_code_snippets {
    (
        @find_start
        r" ```rust"
        $($rest:tt)*
    ) => (
        extract_code_snippets! {
            @accumulate_into []
            $($rest)*
        }
    );

    (
        @find_start
        $otherwise_ignored:tt // ≠ " ```rust"
        $($rest:tt)*
    ) => (
        extract_code_snippets! {
            @find_start
            $($rest)*
        }
    );

    (
        @find_start
        // No lines left
    ) => (
        // The end.
    );

    // End of code snippet found,
    // TIME TO EVALUATE THE CODE!
    (
        @accumulate_into [ $($lines:tt)* ]
        r" ```"
        $($rest:tt)*
    ) => (
        // evaluate the code...
        unstringify!(concat!(
            $($lines),*
        ));
        // ... and rince and repeat with the remaining docstrings
        extract_code_snippets! {
            @find_start
            $($rest)*
        }
    );

    // Basic recursion step: accumulate a non-terminating line
    (
        @accumulate_into [ $($lines:tt)* ]
        $current_line:tt // ≠ " ```"
        $($rest:tt)*
    ) => (
        extract_code_snippets! {
            @accumulate_into [ $($lines)* $current_line ]
            $($rest)*
        }
    );
}

eval_docstrings! {
    /// This is a comment.
    /// As ordinary as they make them.
    ///
    /// And yet...
    /// Sometimes...
    ///
    /// > A code snippet appears!
    ///
    /// ```rust
    /// panic!("Successfully managed to evaluate this panic (and thus panic)");
    /// ```
    ///
    /// Impressive, ain't it?
}

---

Remarks

This intuitive API very quickly encounters limitations, related not the macro itself, but rather to the way Rust expands macros.

So, for instance, the following assertion fails:

assert_eq!(
    stringify!(unstringify!("example")),
    "example",
);

Indeed, in the above code the macro stringify! is called before unstringify!, so what happens is stringify! simply stringifies its input tokens, verbatim, without evaluating them: 'unstringify!("example")'. 🤦

To solve that, unstringify! features "preprocessor" capabilities similar to ::paste::paste!, that allow to circumvent this limitation, by doing:

assert_eq!(
    unstringify!(let $tokens = unstringify!("example") in {
        stringify!($tokens)
    }),
    "example",
);

---

Also, for the same reason but reversed this time, the input fed to unstringify! cannot be eagerly macro-expanded.

This means that the following fails:

macro_rules! my_macro {() => ("fn main () {}")}

unstringify!(my_macro!());

• The workaround is to define things such as my_macro! using, for instance, the callback pattern:

  macro_rules! my_macro {(
      => $callback:ident !
  ) => (
      $callback! { "fn main () {}" }
  )}
  
  my_macro!(=> unstringify!);

That being said, the astute reader may retort:

But wait, doesn't your second example within this documentation showcase unstringify!(stringify! { ... })?

And indeed it does. This is achieved by hard-coding the (basic) logic of stringify! and concat! inside the unstringify! macro (for instance, when unstringify! stumbles upon a stringify! { ... } (which is not, I repeat, a verbatim string literal), it decides to simply emit the inner ...).