Function piston_meta::tokenizer::read_data
source · pub fn read_data(
tokens: &mut Vec<Range<MetaData>>,
range_data: Range<MetaData>,
state: &TokenizerState
) -> TokenizerState
Expand description
Reads meta data.
pub fn read_data(
tokens: &mut Vec<Range<MetaData>>,
range_data: Range<MetaData>,
state: &TokenizerState
) -> TokenizerState
Reads meta data.