Crate utf8_read

UTF8 reader

The utf8-read module provides a streaming char Reader that converts any stream with the std::io::Read into a stream of char values, performing UTF8 decoding incrementally.

If the std::io::Read stream comes from a file then this is just a streaming version of (e.g.) std::fs::read_to_string, but if the it comes from, e.g., a std::net::TcpStream then it has more value: iterating through the characters of the stream will terminate when the TCP stream has stalled mid-UTF8, and can restart when the TCP stream gets more data.

The Reader provided also allows for reading large UTF8 files piecewise; it only reads up to 2kB of data at a time from its stream.

Example

An example use would be:

use utf8_read::Reader;
let str = "This is a \u{1f600} string\nWith a newline\n";
let mut buf_bytes = str.as_bytes();
let mut reader    = Reader::new(&mut buf_bytes);
for x in reader.into_iter() {
    // use char x
}

From a file, one could do:

use utf8_read::Reader;
let in_file = std::fs::File::open("Cargo.toml").unwrap();
let mut reader = Reader::new(&in_file);
for x in reader.into_iter() {
    // use char x
}

Structs

Reader

The Reader provides a stream of characters by UTF-8 decoding a byte stream provided by any type that implements the std::io::Read stream trait.

StreamPosition

This representes the position of a character within a UTF8 stream

Enums

Char

Char represents a unicode character, insufficient data, or an EOF marker

Error

Error represents an error from the UTF-8 character reader, either an IO error from the reader or a malformed UTF-8 encoded set of bytes.

Type Definitions

Result

The Result type is a result with an error type of crate::Error