Expand description
A stream of tokens.
Structural Guarantees
This stream of tokens provides the following structural guarantees.
- Only one space may occur in sequence.
- Only one push may occur in sequence.
- A push may never be preceeded by a line, since it would have no effect.
- Every line must be preceeded by a push.
use genco::Tokens;
use genco::tokens::Item;
let mut tokens = Tokens::<()>::new();
tokens.push();
tokens.push();
assert_eq!(vec![Item::Push::<()>], tokens);
Implementations
Create a new empty stream of tokens.
Examples
use genco::prelude::*;
let tokens = Tokens::<()>::new();
assert!(tokens.is_empty());
Create a new empty stream of tokens with the specified capacity.
Examples
use genco::prelude::*;
let tokens = Tokens::<()>::with_capacity(10);
assert!(tokens.is_empty());
Construct an iterator over the token stream.
Examples
use genco::prelude::*;
use genco::tokens::{ItemStr, Item};
let tokens: Tokens<()> = quote!(foo bar baz);
let mut it = tokens.iter();
assert_eq!(Some(&Item::Literal(ItemStr::Static("foo"))), it.next());
assert_eq!(Some(&Item::Space), it.next());
assert_eq!(Some(&Item::Literal(ItemStr::Static("bar"))), it.next());
assert_eq!(Some(&Item::Space), it.next());
assert_eq!(Some(&Item::Literal(ItemStr::Static("baz"))), it.next());
assert_eq!(None, it.next());
Append the given tokens.
This append function takes anything implementing FormatInto making the argument’s behavior customizable. Most primitive types have built-in implementations of FormatInto treating them as raw tokens.
Most notabley, things implementing FormatInto can be used as arguments for interpolation in the quote! macro.
Examples
use genco::prelude::*;
let mut tokens = Tokens::<()>::new();
tokens.append(4u32);
assert_eq!(quote!(#(4u32)), tokens);
Extend with another stream of tokens.
This respects the structural requirements of adding one element at a time, like you would get by calling space, push, or line.
Examples
use genco::prelude::*;
use genco::tokens::{Item, ItemStr};
let mut tokens: Tokens<()> = quote!(foo bar);
tokens.extend::<Tokens<()>>(quote!(#<space>baz));
assert_eq!(tokens, quote!(foo bar baz));
Walk over all imports.
The order in which the imports are returned is not defined. So if you need them in some particular order you need to sort them.
Examples
use genco::prelude::*;
let debug = rust::import("std::fmt", "Debug");
let ty = rust::import("std::collections", "HashMap");
let tokens = quote!(foo #ty<u32, dyn #debug> baz);
for import in tokens.walk_imports() {
println!("{:?}", import);
}
Add an registered custom element that is not rendered.
Registration can be used to generate imports that do not render a visible result.
Examples
use genco::prelude::*;
let write_bytes_ext = rust::import("byteorder", "WriteBytesExt").with_alias("_");
let tokens = quote!(#(register(write_bytes_ext)));
assert_eq!("use byteorder::WriteBytesExt as _;\n", tokens.to_file_string()?);
Check if tokens contain no items.
use genco::prelude::*;
let tokens: Tokens<()> = quote!();
assert!(tokens.is_empty());
Add a single spacing to the token stream.
Note that due to structural guarantees two consequent spaces may not follow each other in the same token stream.
A space operation has no effect unless it’s followed by a non-whitespace token.
Examples
use genco::prelude::*;
let mut tokens = Tokens::<()>::new();
tokens.space();
tokens.append("hello");
tokens.space();
tokens.space(); // Note: ignored
tokens.append("world");
tokens.space();
assert_eq!(
vec![
" hello world",
],
tokens.to_file_vec()?
);
Add a single push operation.
Push operations ensure that any following tokens are added to their own line.
A push has no effect unless it’s preceeded or followed by non-whitespace tokens.
Examples
use genco::prelude::*;
let mut tokens = Tokens::<()>::new();
tokens.push();
tokens.append("hello");
tokens.push();
tokens.append("world");
tokens.push();
assert_eq!(
vec![
"hello",
"world"
],
tokens.to_file_vec()?
);
Add a single line operation.
A line ensures that any following tokens have one line of separation between them and the preceeding tokens.
A line has no effect unless it’s preceeded and followed by non-whitespace tokens.
Examples
use genco::prelude::*;
let mut tokens = Tokens::<()>::new();
tokens.line();
tokens.append("hello");
tokens.line();
tokens.append("world");
tokens.line();
assert_eq!(
vec![
"hello",
"",
"world"
],
tokens.to_file_vec()?
);
Increase the indentation of the token stream.
An indentation is a language-specific operation which adds whitespace to the beginning of a line preceeding any non-whitespace tokens.
An indentation has no effect unless it’s followed by non-whitespace tokens. It also acts like a push, in that it will shift any tokens to a new line.
Examples
use genco::prelude::*;
let mut tokens = Tokens::<()>::new();
tokens.indent();
tokens.append("hello");
tokens.indent();
tokens.append("world");
tokens.indent();
tokens.append("😀");
assert_eq!(
vec![
" hello",
" world",
" 😀",
],
tokens.to_file_vec()?
);
Decrease the indentation of the token stream.
An indentation is a language-specific operation which adds whitespace to the beginning of a line preceeding any non-whitespace tokens.
An indentation has no effect unless it’s followed by non-whitespace tokens. It also acts like a push, in that it will shift any tokens to a new line.
Indentation can never go below zero, and will just be ignored if that were to happen. However, negative indentation is stored in the token stream, so any negative indentation in place will have to be countered before indentation starts again.
Examples
use genco::prelude::*;
let mut tokens = Tokens::<()>::new();
tokens.indent();
tokens.append("hello");
tokens.unindent();
tokens.append("world");
tokens.unindent();
tokens.append("😀");
tokens.indent();
tokens.append("😁");
tokens.indent();
tokens.append("😂");
assert_eq!(
vec![
" hello",
"world",
"😀",
"😁",
" 😂",
],
tokens.to_file_vec()?
);
Formatting function for token streams that gives full control over the formatting environment.
The configurations and format
arguments will be provided to all
registered language items as well, and can be used to customize
formatting through LangItem::format().
The format
argument is primarily used internally by
Lang::format_file to provide intermediate state that can be affect how
language items are formatter. So formatting something as a file might
yield different results than using this raw formatting function.
Available formatters:
- fmt::VecWriter - To write result into a vector.
- fmt::FmtWriter - To write the result into something implementing fmt::Write.
- fmt::IoWriter- To write the result into something implementing io::Write.
Examples
use genco::prelude::*;
use genco::fmt;
let map = rust::import("std::collections", "HashMap");
let tokens: rust::Tokens = quote! {
let mut m = #map::new();
m.insert(1u32, 2u32);
};
let stdout = std::io::stdout();
let mut w = fmt::IoWriter::new(stdout.lock());
let fmt = fmt::Config::from_lang::<Rust>()
.with_indentation(fmt::Indentation::Space(2));
let mut formatter = w.as_formatter(&fmt);
let config = rust::Config::default();
// Default format state for Rust.
let format = rust::Format::default();
tokens.format(&mut formatter, &config, &format)?;
File formatting function for token streams that gives full control over the formatting environment.
File formatting will render preambles like namespace declarations and imports.
Available formatters:
- fmt::VecWriter - To write result into a vector.
- fmt::FmtWriter - To write the result into something implementing fmt::Write.
- fmt::IoWriter- To write the result into something implementing io::Write.
Examples
use genco::prelude::*;
use genco::fmt;
let map = rust::import("std::collections", "HashMap");
let tokens: rust::Tokens = quote! {
let mut m = #map::new();
m.insert(1u32, 2u32);
};
let stdout = std::io::stdout();
let mut w = fmt::IoWriter::new(stdout.lock());
let fmt = fmt::Config::from_lang::<Rust>()
.with_indentation(fmt::Indentation::Space(2));
let mut formatter = w.as_formatter(&fmt);
let config = rust::Config::default();
tokens.format_file(&mut formatter, &config)?;
Format the token stream as a file for the given target language to a string using the default configuration.
This is a shorthand to using FmtWriter directly in combination with format.
This function will render imports.
Examples
use genco::prelude::*;
use genco::fmt;
let map = rust::import("std::collections", "HashMap");
let tokens: rust::Tokens = quote! {
let mut m = #map::new();
m.insert(1u32, 2u32);
};
assert_eq!(
"use std::collections::HashMap;\n\nlet mut m = HashMap::new();\nm.insert(1u32, 2u32);\n",
tokens.to_file_string()?
);
Format only the current token stream as a string using the default configuration.
This is a shorthand to using FmtWriter directly in combination with format.
This function will not render imports.
Examples
use genco::prelude::*;
let map = rust::import("std::collections", "HashMap");
let tokens: rust::Tokens = quote! {
let mut m = #map::new();
m.insert(1u32, 2u32);
};
assert_eq!(
"let mut m = HashMap::new();\nm.insert(1u32, 2u32);",
tokens.to_string()?
);
Format tokens into a vector, where each entry equals a line in the resulting file using the default configuration.
This is a shorthand to using VecWriter directly in combination with format.
This function will render imports.
Examples
use genco::prelude::*;
let map = rust::import("std::collections", "HashMap");
let tokens: rust::Tokens = quote! {
let mut m = #map::new();
m.insert(1u32, 2u32);
};
assert_eq!(
vec![
"use std::collections::HashMap;",
"",
"let mut m = HashMap::new();",
"m.insert(1u32, 2u32);"
],
tokens.to_file_vec()?
);
Example with Python indentation
use genco::prelude::*;
let tokens: python::Tokens = quote! {
def foo():
pass
def bar():
pass
};
assert_eq!(
vec![
"def foo():",
" pass",
"",
"def bar():",
" pass",
],
tokens.to_file_vec()?
);
Helper function to format tokens into a vector, where each entry equals a line using the default configuration.
This is a shorthand to using VecWriter directly in combination with format.
This function will not render imports.
Examples
use genco::prelude::*;
let map = rust::import("std::collections", "HashMap");
let tokens: rust::Tokens = quote! {
let mut m = #map::new();
m.insert(1u32, 2u32);
};
assert_eq!(
vec![
"let mut m = HashMap::new();",
"m.insert(1u32, 2u32);"
],
tokens.to_vec()?
);
Trait Implementations
Convert the type into tokens in-place. Read more
Formatting a reference to a token stream is exactly the same as extending the token stream with a copy of the stream being formatted.
Examples
use genco::prelude::*;
let a: &Tokens = "e!(foo bar);
let result = quote!(#a baz);
assert_eq!("foo bar baz", result.to_string()?);
Convert the type into tokens in-place. Read more
Creates a value from an iterator. Read more
Creates a value from an iterator. Read more
Construct an owned iterator over the token stream.
Examples
use genco::prelude::*;
use genco::tokens::{ItemStr, Item};
let tokens: Tokens<()> = quote!(foo bar baz);
let mut it = tokens.into_iter();
assert_eq!(Some(Item::Literal(ItemStr::Static("foo"))), it.next());
assert_eq!(Some(Item::Space), it.next());
assert_eq!(Some(Item::Literal(ItemStr::Static("bar"))), it.next());
assert_eq!(Some(Item::Space), it.next());
assert_eq!(Some(Item::Literal(ItemStr::Static("baz"))), it.next());
assert_eq!(None, it.next());
This method returns an ordering between self
and other
values if one exists. Read more
This method tests less than (for self
and other
) and is used by the <
operator. Read more
This method tests less than or equal to (for self
and other
) and is used by the <=
operator. Read more
This method tests greater than (for self
and other
) and is used by the >
operator. Read more
Auto Trait Implementations
impl<L> RefUnwindSafe for Tokens<L> where
L: RefUnwindSafe,
<L as Lang>::Item: RefUnwindSafe,
impl<L> UnwindSafe for Tokens<L> where
L: UnwindSafe,
<L as Lang>::Item: UnwindSafe,
Blanket Implementations
Mutably borrows from an owned value. Read more