Struct rhai::TokenizeState
source · pub struct TokenizeState {
pub max_string_len: Option<NonZeroUsize>,
pub next_token_cannot_be_unary: bool,
pub tokenizer_control: TokenizerControl,
pub comment_level: usize,
pub include_comments: bool,
pub is_within_text_terminated_by: Option<char>,
pub last_token: Option<SmartString<LazyCompact>>,
}
Expand description
(internals) State of the tokenizer.
Exported under the internals
feature only.
Fields§
§max_string_len: Option<NonZeroUsize>
Maximum length of a string.
next_token_cannot_be_unary: bool
Can the next token be a unary operator?
tokenizer_control: TokenizerControl
Shared object to allow controlling the tokenizer externally.
comment_level: usize
Is the tokenizer currently inside a block comment?
include_comments: bool
Include comments?
is_within_text_terminated_by: Option<char>
Is the current tokenizer position within the text stream of an interpolated string?
last_token: Option<SmartString<LazyCompact>>
Textual syntax of the current token, if any.
Set to Some
to begin tracking this information.
Trait Implementations§
source§impl Clone for TokenizeState
impl Clone for TokenizeState
source§fn clone(&self) -> TokenizeState
fn clone(&self) -> TokenizeState
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moresource§impl Debug for TokenizeState
impl Debug for TokenizeState
source§impl Default for TokenizeState
impl Default for TokenizeState
source§fn default() -> TokenizeState
fn default() -> TokenizeState
Returns the “default value” for a type. Read more
source§impl PartialEq<TokenizeState> for TokenizeState
impl PartialEq<TokenizeState> for TokenizeState
source§fn eq(&self, other: &TokenizeState) -> bool
fn eq(&self, other: &TokenizeState) -> bool
This method tests for
self
and other
values to be equal, and is used
by ==
.impl Eq for TokenizeState
impl StructuralEq for TokenizeState
impl StructuralPartialEq for TokenizeState
Auto Trait Implementations§
impl !RefUnwindSafe for TokenizeState
impl !Send for TokenizeState
impl !Sync for TokenizeState
impl Unpin for TokenizeState
impl !UnwindSafe for TokenizeState
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more