pub fn detect( tokens: &[Token], dialect: &Dialect, ) -> (Vec<Unit>, Vec<Diagnostic>)
Detect unit boundaries and recoverable boundary errors under dialect.
dialect