Crate litext 

litext

A procedural macro library for extracting string literal contents from tokens. Built for proc-macro authors who need to pull string content from TokenStream input.

Overview

This library provides the extract and extract_litstr functions, as well as the litext! macro which is the primary interface for most use cases. It handles both regular string literals and raw string literals.

This is a proc-macro helper library. It is designed for proc-macro authors who need to extract string content from tokens during macro expansion. It is not intended for use in regular Rust code.

The implementation is zero dependency and contained in a single file.

Usage

Use in your proc-macro. Pass the raw tokens to litext!:

use litext::{litext, TokenStream};

pub fn my_macro(input: TokenStream) -> TokenStream {
    let string_content = litext!(input); // or litext!(input as String); to be explicit
    // ... use string_content
}

To also capture the source span of the literal, use the as LitStr form:

use litext::{litext, TokenStream};

pub fn my_macro(input: TokenStream) -> TokenStream {
    let lit = litext!(input as LitStr);
    // lit.value() for the string, lit.span() for the source location
}

Features

The following string literal formats are supported:

• Regular double-quoted strings: "hello world"
• Raw strings with hashes: r#"hello world"#
• Raw strings with multiple hashes: r##"hello # world"##

Errors are returned for:

• Empty input: litext! expected a string literal, got nothing
• Multiple tokens: litext! expected exactly one string literal
• Not a string literal: litext! expected a string literal, found ...
• Byte strings: litext: expected a string literal, not a byte string

Performance

The macro and extract functions run at compile time during macro expansion. There is no runtime overhead in your final binary.

Re-exports

This crate re-exports proc_macro::TokenStream, proc_macro::TokenTree, proc_macro::Literal, proc_macro::Span, and LitStr for convenience in proc-macro code.

Macros

litext

The whole point.

Structs

LitStr

A parsed string literal, bundling the extracted text with its source location.

Literal

A literal string ("hello"), byte string (b"hello"), C string (c"hello"), character ('a'), byte character (b'a'), an integer or floating point number with or without a suffix (1, 1u8, 2.3, 2.3f32). Boolean literals like true and false do not belong here, they are Idents.

Span

A region of source code, along with macro expansion information.

TokenStream

The main type provided by this crate, representing an abstract stream of tokens, or, more specifically, a sequence of token trees. The type provides interfaces for iterating over those token trees and, conversely, collecting a number of token trees into one stream.

Enums

TokenTree

A single token or a delimited sequence of token trees (e.g., [1, (), ..]).

Functions

extract

Extracts the inner text content from a token stream representing a string literal.

extract_litstr

Extracts the inner text content from a token stream, returning a LitStr that bundles the string value with its source span.