sylt-tokenizer-0.1.0
- sylt-tokenizer 0.1.0
- Docs.rs crate page
- MIT
- Links
- Repository
- crates.io
- Source
- Owners
- sornas
- Dependencies
- logos ^0.12 normal
- Versions
- 3.9% of the crate is documented
Platform
Feature flags

docs.rs

Rust

string_to_tokens

sylt_tokenizer0.1.0

Function string_to_tokens

pub fn string_to_tokens(content: &str) -> Vec<PlacedToken>