litext

Literal extraction for proc-macro authors. Pull typed values out of TokenStream input without the boilerplate.

let value: String = litext!(input);
let count: u32     = litext!(input as u32);
let (s, n): (String, i32) = litext!(input as (String , i32));

What is this?

When you write a procedural macro, you often receive a TokenStream that contains a single literal and need to pull out its value. Doing this correctly means handling every literal kind, every radix, escape sequences, raw strings, byte strings, C strings, and all the edge cases. litext does all that for you.

The library supports every Rust literal kind, span-aware wrapper types for precise diagnostics, a ToTokens round-trip trait, a FromLit extension point for custom types, and a variadic multi-extraction form for pulling several values out of one TokenStream at once.

Installation

cargo add litext

Your proc-macro entry point needs to bridge proc_macro and proc_macro2:

#[proc_macro]
pub fn my_macro(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
    my_macro_inner(input.into()).into()
}

fn my_macro_inner(input: proc_macro2::TokenStream) -> proc_macro2::TokenStream {
    let s: String = litext!(input);
    // ...
}

Quick Start

Extract a string literal:

use litext::{litext, TokenStream};

fn my_macro(input: TokenStream) -> TokenStream {
    let text: String = litext!(input);
    // Returns early with a compile error if extraction fails.
    // Use `litext!(input as String)` for the same result, explicitly typed.
}

Extract other literal types:

let num: i32  = litext!(input as i32);
let val: f64  = litext!(input as f64);
let ch: char  = litext!(input as char);
let flag: bool = litext!(input as bool);

Keep the Result instead of returning early:

let result: Result<String, TokenStream> = litext!(try input);
match result {
    Ok(text) => { /* use text */ }
    Err(e)   => return e,
}

// Same with an explicit type:
let result: Result<i32, TokenStream> = litext!(try input as i32);

Extracting Multiple Literals at Once

litext! can extract several values from one TokenStream in a single call. Wrap the types in a tuple and put the separator token between them:

// Input TokenStream: "hello" , 42
let (s, n): (String, i32) = litext!(input as (String , i32));

// Three values separated by commas:
// Input: "hello" , 3.14 , true
let (s, f, b): (String, f64, bool) = litext!(input as (String , f64 , bool));

// Semicolon separator:
// Input: "key" ; 100
let (key, val): (String, u32) = litext!(input as (String ; u32));

The try form also works for tuples:

let result: Result<(String, i32), _> = litext!(try input as (String , i32));

Up to 12 values can be extracted in one call.

Separator rules: The separator written in the macro call and the separator in the actual TokenStream must be the same single punctuation character. Any single ASCII punctuation character works: ,, ;, |, &, +, -, *, /, %, ^, !, ?, <, >, =, ., :, @, ~.

Multi-character punctuation sequences such as ->, =>, and :: do not work as separators. Rust tokenizes them as multiple tokens, so only the first character would be consumed and the extraction would fail. Grouping delimiters ((, ), [, ], {, }) and $ are also not valid separators.

Note: Type arguments in tuple position must be single-token identifiers. Generic types like LitInt<u8> span multiple tokens and cannot appear in this position. Use the default forms (LitInt, LitFloat) which carry their default type parameters.

Span-Aware Types

Every literal kind has a span-aware wrapper that bundles the parsed value with its source location. Use these when you need to emit a compiler diagnostic pointing at the exact literal in the user's code.

use litext::{litext, TokenStream};
use litext::literal::LitStr;

fn my_macro(input: TokenStream) -> TokenStream {
    let lit: LitStr = litext!(input as LitStr);

    if lit.value().is_empty() {
        return comperr::error(lit.span(), "string cannot be empty");
    }

    // Use lit.value() for the content, lit.span() for diagnostics.
}

use litext::literal::LitInt;

let lit: LitInt<u8> = litext!(input as LitInt<u8>);
let value: u8 = *lit.value();

if value == 0 {
    return comperr::error(lit.span(), "value cannot be zero");
}

All span-aware types expose:

• .value() - the parsed value
• .span() - the source location
• .suffix() - the explicit type suffix if present ("i32" for 42i32, None for 42)

Round-Tripping with ToTokens

All span-aware types implement ToTokens, which converts them back into a TokenStream. This enables an extract, validate, emit pattern:

use litext::literal::{LitStr, ToTokens};

let lit: LitStr = litext!(input as LitStr);

if lit.value().starts_with("_") {
    return comperr::error(lit.span(), "string cannot start with underscore");
}

// Emit the literal back into the output with its original span preserved.
lit.to_token_stream()

Custom Literal Types

Implement FromLit to make litext!(input as MyType) work for your own types:

use litext::literal::FromLit;
use proc_macro2::{Literal, TokenStream};

pub struct NonEmpty(String);

impl FromLit for NonEmpty {
    fn from_lit(lit: Literal) -> Result<Self, TokenStream> {
        let s = String::from_lit(lit)?;
        if s.is_empty() {
            return Err(comperr::error(
                proc_macro2::Span::call_site(),
                "string cannot be empty",
            ));
        }
        Ok(NonEmpty(s))
    }
}

// Then in your macro:
let val: NonEmpty = litext!(input as NonEmpty);

Override from_ident if your type is represented as an identifier token (like bool):

fn from_ident(ident: proc_macro2::Ident) -> Result<Self, TokenStream> {
    // handle true/false/etc.
}

What Is Supported

Strings

Integers

All integer types are supported: i8 to i128, isize, u8 to u128, usize. Overflow returns a compile error.

Floats

Characters

Full escape support: 'a', '\n', '\t', '\\', '\'', '\"', '\0', '\x41', '\u{1F600}'.

Booleans

true and false are parsed as identifier tokens, not literals. All litext extraction handles this transparently.

Byte Literals

b'a', b'\n', b'\xff'. The full 0x00..=0xFF range is valid for \x escapes in byte literals.

Byte Strings

b"hello", b"\xff\x80", br#"raw bytes"#. The full byte range is supported for \x escapes.

C Strings

c"hello", cr#"raw"#. Interior null bytes are rejected with a compile error.

Error Handling

All errors are returned as TokenStream values containing compile_error! invocations. When returned from a proc-macro entry point, the compiler displays the error at the correct source location.

Negative numbers (-42) are two tokens in Rust's token stream: a - punctuation token followed by a positive literal. litext does not support them. Parse the sign and magnitude separately if needed.

Planned Features

Planned

• Feature flag gates to FromLit impls for standard types, makes litext lighter (2.0)
• Multi-character separators if possible (<=2.0)
• Negative numbers maybe (<=2.0)

Requirements

• Rust 2024 edition (Rust 1.85)
• proc_macro2 as a direct dependency in your crate

License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Cheers, RazkarStudio

© 2026 RazkarStudio. All rights reserved.