Struct yap::types::IterTokens
source · pub struct IterTokens<I> { /* private fields */ }
Expand description
This is what we are given back if we call [IterTokens::into_tokens(iter)
] on
an impl Iterator + Clone
. It implements the Tokens
interface.
Implementations§
source§impl<I> IterTokens<I>
impl<I> IterTokens<I>
sourcepub fn new(iter: I) -> Self
pub fn new(iter: I) -> Self
We can’t define a blanket impl for IntoTokens
on all impl Iterator + Clone
without
specialization.
Instead, you must manually construct new IterTokens
using this function.
Example
use yap::{ Tokens, types::IterTokens };
// In normal usage, "hello \n\t world".into_tokens()
// would be preferred here (which would give StrTokens).
// This is just to demonstrate using IterTokens:
let chars_iter = "hello \n\t world".chars();
let mut tokens = IterTokens::new(chars_iter);
let loc = tokens.location();
// now we have tokens, we can do some parsing:
assert!(tokens.tokens("hello".chars()));
tokens.skip_while(|c| c.is_whitespace());
assert!(tokens.tokens("world".chars()));
// We can reset the location too as with other Tokens impls.
// A location here is just a copy of the iterator at an
// earlier point.
tokens.set_location(loc);
assert!(tokens.tokens("hello".chars()));
sourcepub fn into_inner(self) -> I
pub fn into_inner(self) -> I
Return the inner iterator, consuming self.
Trait Implementations§
source§impl<I: Clone> Clone for IterTokens<I>
impl<I: Clone> Clone for IterTokens<I>
source§fn clone(&self) -> IterTokens<I>
fn clone(&self) -> IterTokens<I>
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moresource§impl<I> IntoTokens<<I as Iterator>::Item> for IterTokens<I>where
I: Iterator + Clone,
impl<I> IntoTokens<<I as Iterator>::Item> for IterTokens<I>where I: Iterator + Clone,
§type Tokens = IterTokens<I>
type Tokens = IterTokens<I>
The type that will be used to implement the
Tokens
interface.source§fn into_tokens(self) -> Self
fn into_tokens(self) -> Self
Convert self into a type which implements the
Tokens
interface.source§impl<I> Tokens for IterTokens<I>where
I: Iterator + Clone,
impl<I> Tokens for IterTokens<I>where I: Iterator + Clone,
§type Item = <I as Iterator>::Item
type Item = <I as Iterator>::Item
The item returned from
Tokens::next()
.§type Location = IterTokensLocation<I>
type Location = IterTokensLocation<I>
An object which can be used to reset the token stream
to some position.
source§fn next(&mut self) -> Option<Self::Item>
fn next(&mut self) -> Option<Self::Item>
Return the next token. This is also the basis of the
Iterator
implementation
that’s returned when you call Tokens::as_iter()
. By implementing it here, we can keep
all of the methods provided by Iterator
in a separate “namespace” to avoid confusion
and potential name collisions. Read moresource§fn location(&self) -> Self::Location
fn location(&self) -> Self::Location
Return a “location” pointer. This can be passed to
Tokens::set_location
to set the tokens location back to the state at the time it was handed out.
If the crate::TokenLocation
trait is in scope, you can also call the
crate::TokenLocation::offset()
method on it to obtain the current offset. Read moresource§fn set_location(&mut self, location: Self::Location)
fn set_location(&mut self, location: Self::Location)
Set the tokens to the location provided. See
Tokens::location
.source§fn is_at_location(&self, location: &Self::Location) -> bool
fn is_at_location(&self, location: &Self::Location) -> bool
Return true if the current cursor location matches the location given, or false
otherwise. Read more
source§fn offset(&self) -> usize
fn offset(&self) -> usize
Return the current offset into the tokens that we’ve parsed up to so far.
The exact meaning of this can vary by implementation; when parsing slices, it
is index of the slice item we’ve consumed up to, and when
parsing
&str
’s it is the number of bytes (not characters) consumed so far. Read moresource§fn as_iter(&mut self) -> TokensAsIter<'_, Self>
fn as_iter(&mut self) -> TokensAsIter<'_, Self>
Return an iterator over our tokens. The
Tokens
trait already mirrors the Iterator
interface by providing Tokens::Item
and Tokens::next()
, but we keep the Iterator
separate to avoid collisions, and because some iterator methods don’t consume tokens as you
might expect, and so must be used with care when parsing input.source§fn into_iter(self) -> TokensIntoIter<Self>
fn into_iter(self) -> TokensIntoIter<Self>
Like
Tokens::as_iter()
, except it consumes self
, which can be useful in some situations.source§fn parse<Out, Buf>(&mut self) -> Result<Out, <Out as FromStr>::Err>where
Out: FromStr,
Buf: FromIterator<Self::Item> + Deref<Target = str>,
fn parse<Out, Buf>(&mut self) -> Result<Out, <Out as FromStr>::Err>where Out: FromStr, Buf: FromIterator<Self::Item> + Deref<Target = str>,
Attempt to parse the remaining tokens into the first
Out
generic using str::parse()
.
The second generic type may be used to buffer tokens, and can be any type that implements
FromIterator<Self::Item> + Deref<Target = str>
. Read moresource§fn with_context<C>(self, context: C) -> WithContext<Self, C>
fn with_context<C>(self, context: C) -> WithContext<Self, C>
Attach some context to your tokens. The returned struct,
WithContext
, also implements
Tokens
, and so has can be used in much the same way. Since this consumes your tokens, it’s
better suited to permanent context that you’d like throughout the parsing. Read moresource§fn with_context_mut<C>(&mut self, context: C) -> WithContextMut<&mut Self, C>
fn with_context_mut<C>(&mut self, context: C) -> WithContextMut<&mut Self, C>
Unlike
Tokens::with_context
, which consumes the tokens, this borrows them mutably, allowing it to
be used when you only have a mutable reference to tokens (which is a common function signature to use),
and making it better suited to attaching temporary contexts. Read moresource§fn slice(&mut self, from: Self::Location, to: Self::Location) -> Slice<'_, Self>
fn slice(&mut self, from: Self::Location, to: Self::Location) -> Slice<'_, Self>
Return a slice of tokens starting at the
to
location provided and ending just prior to
the from
location provided (ie equivalent to the range to..from
). Read moresource§fn peek(&mut self) -> Option<Self::Item>
fn peek(&mut self) -> Option<Self::Item>
Return the next item in the input without consuming it. Read more
source§fn take_while<F>(&mut self, f: F) -> TakeWhile<'_, Self, F>where
F: FnMut(&Self::Item) -> bool,
fn take_while<F>(&mut self, f: F) -> TakeWhile<'_, Self, F>where F: FnMut(&Self::Item) -> bool,
source§fn skip_while<F>(&mut self, f: F) -> usizewhere
F: FnMut(&Self::Item) -> bool,
fn skip_while<F>(&mut self, f: F) -> usizewhere F: FnMut(&Self::Item) -> bool,
Iterate over the tokens until the provided function returns false on one. Only consume the tokens
that the function returned true for, returning the number of tokens that were consumed/skipped.
Equivalent to
toks.take_while(f).count()
. Read moresource§fn many<F, Output>(&mut self, parser: F) -> Many<'_, Self, F>where
F: FnMut(&mut Self) -> Option<Output>,
fn many<F, Output>(&mut self, parser: F) -> Many<'_, Self, F>where F: FnMut(&mut Self) -> Option<Output>,
source§fn many_err<F, Output, E>(&mut self, parser: F) -> ManyErr<'_, Self, F>where
F: FnMut(&mut Self) -> Result<Output, E>,
fn many_err<F, Output, E>(&mut self, parser: F) -> ManyErr<'_, Self, F>where F: FnMut(&mut Self) -> Result<Output, E>,
source§fn skip_many<F>(&mut self, parser: F) -> usizewhere
F: FnMut(&mut Self) -> bool,
fn skip_many<F>(&mut self, parser: F) -> usizewhere F: FnMut(&mut Self) -> bool,
Ignore 0 or more instances of some parser. Read more
source§fn skip_many1<F, E, Ignored>(&mut self, parser: F) -> Result<usize, E>where
F: FnMut(&mut Self) -> Result<Ignored, E>,
fn skip_many1<F, E, Ignored>(&mut self, parser: F) -> Result<usize, E>where F: FnMut(&mut Self) -> Result<Ignored, E>,
Ignore 1 or more instances of some parser. If the provided parser
fails immediately, return the error that it produced. Read more
source§fn sep_by<F, S, Output>(
&mut self,
parser: F,
separator: S
) -> SepBy<'_, Self, F, S>where
F: FnMut(&mut Self) -> Option<Output>,
S: FnMut(&mut Self) -> bool,
fn sep_by<F, S, Output>( &mut self, parser: F, separator: S ) -> SepBy<'_, Self, F, S>where F: FnMut(&mut Self) -> Option<Output>, S: FnMut(&mut Self) -> bool,
source§fn sep_by_err<F, S, E, Output>(
&mut self,
parser: F,
separator: S
) -> SepByErr<'_, Self, F, S>where
F: FnMut(&mut Self) -> Result<Output, E>,
S: FnMut(&mut Self) -> bool,
fn sep_by_err<F, S, E, Output>( &mut self, parser: F, separator: S ) -> SepByErr<'_, Self, F, S>where F: FnMut(&mut Self) -> Result<Output, E>, S: FnMut(&mut Self) -> bool,
Return a
Tokens
impl that parses anything matching the parser
function, and expects
to parse something matching the separator
function between each one. Unlike Tokens::sep_by
,
this accepts parsers that return Result
s, and returns the result on each iteration. Once
an error is hit, None
is returned thereafter. Read moresource§fn sep_by_all<F, S, Output>(
&mut self,
parser: F,
separator: S
) -> SepByAll<'_, Self, F, S, Output>where
F: FnMut(&mut Self) -> Option<Output>,
S: FnMut(&mut Self) -> Option<Output>,
fn sep_by_all<F, S, Output>( &mut self, parser: F, separator: S ) -> SepByAll<'_, Self, F, S, Output>where F: FnMut(&mut Self) -> Option<Output>, S: FnMut(&mut Self) -> Option<Output>,
Returns a
Tokens
impl that parses anything matching the parser
function,
and expects to parse something matching the separator
function between each one.
The Tokens
impl hands back the output from both the parser
and separator
function, which means that they are both expected to return the same type. Read moresource§fn sep_by_all_err<F, S, Output, E>(
&mut self,
parser: F,
separator: S
) -> SepByAllErr<'_, Self, F, S, Output>where
F: FnMut(&mut Self) -> Result<Output, E>,
S: FnMut(&mut Self) -> Option<Output>,
fn sep_by_all_err<F, S, Output, E>( &mut self, parser: F, separator: S ) -> SepByAllErr<'_, Self, F, S, Output>where F: FnMut(&mut Self) -> Result<Output, E>, S: FnMut(&mut Self) -> Option<Output>,
Similar to
Tokens::sep_by_all
, except that the Tokens
impl returned also
hands back the first error encountered when attempting to run our parser
. Read moresource§fn surrounded_by<F, S, Output>(&mut self, parser: F, surrounding: S) -> Outputwhere
F: FnOnce(&mut Self) -> Output,
S: FnMut(&mut Self),
fn surrounded_by<F, S, Output>(&mut self, parser: F, surrounding: S) -> Outputwhere F: FnOnce(&mut Self) -> Output, S: FnMut(&mut Self),
Parse some tokens that are optionally surrounded by the result of a
surrounding
parser. Read moresource§fn optional<F, Output>(&mut self, f: F) -> Outputwhere
F: FnOnce(&mut Self) -> Output,
Output: IsMatch,
fn optional<F, Output>(&mut self, f: F) -> Outputwhere F: FnOnce(&mut Self) -> Output, Output: IsMatch,
Attempt to parse some output from the tokens. Returning
None
or false
signifies that no match was found, and no tokens will
be consumed. Otherwise, we’ll return the match and consume the tokens. Read moresource§fn optional_err<F, Output, Error>(&mut self, f: F) -> Result<Output, Error>where
F: FnOnce(&mut Self) -> Result<Output, Error>,
fn optional_err<F, Output, Error>(&mut self, f: F) -> Result<Output, Error>where F: FnOnce(&mut Self) -> Result<Output, Error>,
Attempt to parse some output from the tokens, returning a
Result
.
If the Result
returned is Err
, no tokens will be consumed. Read moresource§fn consume(&mut self)
fn consume(&mut self)
Consume all remaining tokens. This is expected to be used in conjunction
with combinators like
Tokens::take
and Tokens::take_while
. This is
just a shorthand for toks.as_iter().for_each(drop)
. Read moresource§fn collect<B: FromIterator<Self::Item>>(&mut self) -> B
fn collect<B: FromIterator<Self::Item>>(&mut self) -> B
Collect up all of the tokens into something that implements
FromIterator
. If you’d like to call str::parse
on the
subsequent collection, then prefer Tokens::parse
, which can
be more optimal in some cases. Read moreAuto Trait Implementations§
impl<I> RefUnwindSafe for IterTokens<I>where I: RefUnwindSafe,
impl<I> Send for IterTokens<I>where I: Send,
impl<I> Sync for IterTokens<I>where I: Sync,
impl<I> Unpin for IterTokens<I>where I: Unpin,
impl<I> UnwindSafe for IterTokens<I>where I: UnwindSafe,
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more