litext

A lightweight procedural macro library for extracting string literal contents from tokens, because doing that shouldn't bloat compilation time.

let value: String = litext!(input);

What is this?

Litext provides a macro to extract the inner text from a string literal token. You pass raw tokens to litext! and it gives you back the unwrapped string content without the quotes, optionally with the source span attached for precise diagnostics.

This is a proc-macro helper library. It is designed for proc-macro authors who need to extract string content from TokenStream input during macro expansion without the bloat.

Zero dependencies, tiny, built for proc-macro authors.

Installation

Add litext to your project:

cargo add litext

Quick Start

Extract the string content as a String:

use litext::{litext, TokenStream};

pub fn my_macro(input: TokenStream) -> TokenStream {
    let content: String = litext!(input);
    // or, to be explicit:
    let content: String = litext!(input as String);
    // ... use content
}

Extract with span information for precise diagnostics:

use litext::{litext, LitStr, TokenStream};

pub fn my_macro(input: TokenStream) -> TokenStream {
    let lit: LitStr = litext!(input as LitStr);
    let value = lit.value(); // the string content
    let span = lit.span();   // where it appeared in source
    // ... use value and span
}

Handle errors with the try variant (returns Result instead of early return):

use litext::{litext, TokenStream};

pub fn my_macro(input: TokenStream) -> TokenStream {
    let result = litext!(try input);
    match result {
        Ok(text) => { /* use text */ }
        Err(e) => e, // Forward the error TokenStream
    }
}

Return Result but for other types:

let result = litext!(try input as Type);

Extract other literal types directly:

use litext::{litext, TokenStream};

pub fn my_int_macro(input: TokenStream) -> TokenStream {
    let num: i32 = litext!(input as i32);       // integer
    let val: f64 = litext!(input as f64);       // float
    let ch: char = litext!(input as char);      // character
    let flag: bool = litext!(input as bool);    // boolean
}

Span-Aware Types

For precise error reporting, use the span-aware types:

use litext::{litext, TokenStream};
use litext::literal::{LitInt, LitFloat, LitBool, LitChar};

pub fn my_macro(input: TokenStream) -> TokenStream {
    let lit: LitInt<u64> = litext!(input as LitInt<u64>);
    let value: u64 = lit.value;
    let span = lit.span(); // Use for error reporting

    if value == 0 {
        return comperr::error(span, "value cannot be zero");
    }
}

Creating Your Own Literal Type

Implement the FromLit trait to support custom types:

use litext::literal::FromLit;
use proc_macro2::{Literal, TokenStream};

pub struct MyType(String);

impl FromLit for MyType {
    fn from_lit(lit: Literal) -> Result<Self, TokenStream> {
        // Parse the literal however you need
        Ok(MyType(lit.to_string()))
    }
}

// Then use it:
// let my: MyType = litext!(input as MyType);

Features

Literal Extraction

• Extract string content from TokenStream tokens
• Support for regular strings: "hello world" with full escape sequences (\n, \r, \t, \\, \", \0, \x41, \u{1F600})
• Support for raw strings: r#"hello world"# (no escape processing)
• Support for raw strings with multiple hashes: r##"hello #" world"##
• Integer literals with all radixes: 42, 0xFF (hex), 0o77 (octal), 0b1010 (binary)
• Integer literals with underscore separators: 1_000_000
• Integer literals with type suffixes: 42_i32, 100u8
• Float literals: 3.14, 1e10, 2.5e-3 (scientific notation)
• Float literals with underscore separators: 1_000.5
• Float literals with type suffixes: 3.14_f32
• Character literals with escape sequences: 'x', '\n', '\u{1F600}', '\x41'
• Boolean literals: true, false (as identifiers)
• Byte literals: b'a', b'\n', b'\xff'
• Byte string literals: b"hello", br#"raw"#
• C string literals: c"hello", cr#"raw"# (null byte validated)

Error Handling

• try variant returns Result<T, TokenStream> for explicit error handling: litext!(try input)
• try variant with custom types: litext!(try input as T)
• Default variant returns early on error for concise code: litext!(input as T)
• Span-accurate error reporting via comperr integration

Span-Aware Types

• LitStr - String literal with span
• LitInt<T> - Integer literal with span (supports i8–i128, u8–u128, isize, usize; defaults to i32)
• LitFloat<T> - Float literal with span (supports f32, f64; defaults to f64)
• LitBool - Boolean literal with span
• LitChar - Character literal with span
• LitByte - Byte literal with span
• LitByteStr - Byte string literal with span
• LitCStr - C string literal with span

Round-Tripping

• ToTokens trait for converting span-aware types back to TokenStream
• Enables extract → validate → emit patterns in proc-macros

Extensibility

• FromLit trait for implementing custom literal extraction
• Create your own literal types by implementing FromLit
• FromIdent support for identifier-based literals (like bool)

Limitations

• Negative numbers like -42 are not literals in Rust's token stream -- they are two tokens (a - punct and a positive literal). litext does not support them. Handle negation at the expression level instead.

Other

• Clear error messages for invalid inputs
• Dependencies: proc_macro2 (for non-proc-macro contexts), comperr (for error reporting)
• Built for proc-macro authors

Requirements

• Rust 2024 edition
• A proc-macro crate (this library is for macro authors, not end users)

License

Licensed under either of:

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Cheers, RazkarStudio

© 2026 RazkarStudio. All rights reserved.