Crate litrs

source ·
Expand description

Parsing and inspecting Rust literal tokens.

This library offers functionality to parse Rust literals, i.e. tokens in the Rust programming language that represent fixed values. The grammar for those is defined here.

This kind of functionality already exists in the crate syn. However, as you oftentimes don’t need (nor want) the full power of syn, litrs was built. This crate also offers a bit more flexibility compared to syn (only regarding literals, of course).

Quick start

StringLit::try_from(tt)?.value()

… where tt is a proc_macro::TokenTree and where StringLit can be replaced with Literal or other types of literals (e.g. FloatLit). Calling value() returns the value that is represented by the literal.

Mini Example

ⓘ
use proc_macro::TokenStream;

#[proc_macro]
pub fn foo(input: TokenStream) -> TokenStream {
     let first_token = input.into_iter().next().unwrap(); // Do proper error handling!
     let string_value = match litrs::StringLit::try_from(first_token) {
         Ok(string_lit) => string_lit.value(),
         Err(e) => return e.to_compile_error(),
     };

     // `string_value` is the string value with all escapes resolved.
     todo!()
}

Overview

The main types of this library are Literal, representing any kind of literal, and *Lit, like StringLit or FloatLit, representing a specific kind of literal.

There are different ways to obtain such a literal type:

  • parse: parses a &str or String and returns Result<_, ParseError>. For example: Literal::parse and IntegerLit::parse.

  • From<proc_macro::Literal> for Literal: turns a Literal value from the proc_macro crate into a Literal from this crate.

  • TryFrom<proc_macro::Literal> for *Lit: tries to turn a proc_macro::Literal into a specific literal type of this crate. If the input is a literal of a different kind, Err(InvalidToken) is returned.

  • TryFrom<proc_macro::TokenTree>: attempts to turn a token tree into a literal type of this crate. An error is returned if the token tree is not a literal, or if you are trying to turn it into a specific kind of literal and the token tree is a different kind of literal.

All of the From and TryFrom conversions also work for reference to proc_macro types. Additionally, if the crate feature proc-macro2 is enabled (which it is by default), all these From and TryFrom impls also exist for the corresponding proc_macro2 types.

Note: true and false are Idents when passed to your proc macro. The TryFrom<TokenTree> impls check for those two special idents and return a BoolLit appropriately. For that reason, there is also no TryFrom<proc_macro::Literal> impl for BoolLit. The proc_macro::Literal simply cannot represent bool literals.

Examples

In a proc-macro:

ⓘ
use std::convert::TryFrom;
use proc_macro::TokenStream;
use litrs::FloatLit;

#[proc_macro]
pub fn foo(input: TokenStream) -> TokenStream {
     let mut input = input.into_iter().collect::<Vec<_>>();
     if input.len() != 1 {
         // Please do proper error handling in your real code!
         panic!("expected exactly one token as input");
     }
     let token = input.remove(0);

     match FloatLit::try_from(token) {
         Ok(float_lit) => { /* do something */ }
         Err(e) => return e.to_compile_error(),
     }

     // Dummy output
     TokenStream::new()
}

Parsing from string:

use litrs::{FloatLit, Literal};

// Parse a specific kind of literal (float in this case):
let float_lit = FloatLit::parse("3.14f32");
assert!(float_lit.is_ok());
assert_eq!(float_lit.unwrap().suffix(), "f32");
assert!(FloatLit::parse("'c'").is_err());

// Parse any kind of literal. After parsing, you can inspect the literal
// and decide what to do in each case.
let lit = Literal::parse("0xff80").expect("failed to parse literal");
match lit {
    Literal::Integer(lit) => { /* ... */ }
    Literal::Float(lit) => { /* ... */ }
    Literal::Bool(lit) => { /* ... */ }
    Literal::Char(lit) => { /* ... */ }
    Literal::String(lit) => { /* ... */ }
    Literal::Byte(lit) => { /* ... */ }
    Literal::ByteString(lit) => { /* ... */ }
}

Crate features

  • proc-macro2 (default): adds the dependency proc_macro2, a bunch of From and TryFrom impls, and InvalidToken::to_compile_error2.
  • check_suffix: if enabled, parse functions will exactly verify that the literal suffix is valid. Adds the dependency unicode-xid. If disabled, only an approximate check (only in ASCII range) is done. If you are writing a proc macro, you don’t need to enable this as the suffix is already checked by the compiler.

Structs

  • ByteLit
    A (single) byte literal, e.g. b'k' or b'!'.
  • ByteStringLit
    A byte string or raw byte string literal, e.g. b"hello" or br#"abc"def"#.
  • CharLit
    A character literal, e.g. 'g' or '🦊'.
  • FloatLit
    A floating point literal, e.g. 3.14, 8., 135e12, or 1.956e2f64.
  • IntegerLit
    An integer literal, e.g. 27, 0x7F, 0b101010u8 or 5_000_000i64.
  • InvalidToken
    An error signaling that a different kind of token was expected. Returned by the various TryFrom impls.
  • ParseError
    Errors during parsing.
  • StringLit
    A string or raw string literal, e.g. "foo", "Grüße" or r#"a🦊c"d🦀f"#.

Enums

Traits

  • Buffer
    A shared or owned string buffer. Implemented for String and &str. Implementation detail.
  • FromIntegerLiteral
    Integer literal types. Implementation detail.