Crate basic_text

Basic Text strings and I/O streams

This crate provides several utilities for working with Basic Text.

• TextString and TextStr are similar to the standard library’s String and str, but use the Basic Text string format, along with a text!("...") macro for Basic Text string literals.

• TextReader and TextWriter are input and output streams which use the Basic Text stream format. On input, content is converted in a way which is lossy with respect to the original bytestream. Output uses the “strict” conversion method, in which invalid content is diagnosed with errors.

• BufReadText, an extension trait that adds text_lines and text_lines_lossy to BufRead implementations for reading lines from an input stream as BasicText strings.

• TextDuplexer is a Duplex for reading and writing on an interactive stream using Basic Text.

Examples

Working with TextString and company is overall similar to working with String and company, but with a little more syntax in some places:

use basic_text::{text, text_substr, ReadText, TextReader, TextString, TextWriter, WriteText};
use std::io::{stdin, stdout, Write};

// Wrap stdout in an output stream that ensures that the output is
// Basic Text.
let mut stream = TextWriter::new(stdout());

// Construct Basic Text literals.
let greeting = text!("Hello, World!");

// Write Basic Text directly.
stream.write_text(greeting).unwrap();

// `TextString` can't be split at arbitrary boundaries, so this crate has
// substring types, so you can work with Basic Text content incrementally.
// The following code prints the "Service Dog" ZWJ Sequence "🐕‍🦺" in
// parts, where splitting it would not be valid in Basic Text.
stream
    .write_text_substr(text_substr!("🐕\u{200d}"))
    .unwrap();
stream.write_text_substr(text_substr!("🦺")).unwrap();

// Regular strings with Basic Text content can be written.
writeln!(stream, "Valid!").unwrap();

// But invalid content is diagnosed as an error.
writeln!(stream, "\u{c}Invalid!\u{7}").unwrap_err();

// A Basic Text reader, on the other hand, always succeeds, by replacing
// invalid sequences with `�`s.
let mut s = TextString::new();
TextReader::new(stdin())
    .read_to_text_string(&mut s)
    .unwrap();

Macros

text

TextStr literal support: text!("string literal").

text_substr

TextSubstr literal support: text_substr!("string literal\u{200e}").

Structs

FromTextError

FromTextError is to TextString as FromUtf8Error is to String.

TextDuplexer

A HalfDuplex implementation which translates from an input HalfDuplex implementation producing an arbitrary byte sequence into a valid Basic Text stream.

TextError

TextError is to TextString as Utf8Error is to String.

TextLines

An iterator over the lines of an instance of BufReadText.

TextLinesLossy

An iterator over the lines of an instance of BufReadText.

TextReader

A Read implementation which translates from an input Read implementation producing an arbitrary byte sequence into a valid Basic Text stream.

TextStr

Text slices.

TextString

A Basic Text encoded, growable string.

TextSubstr

Text substring slices.

TextSubstring

A substring of a Basic Text string or stream.

TextWriter

A WriteLayered implementation which translates to an output WriteLayered producing a valid Basic Text stream from an arbitrary byte sequence.

Constants

NORMALIZATION_BUFFER_SIZE

The minimum size of a buffer needed to perform NFC normalization, and thus the minimum size needed to pass to TextReader’s read.

UNICODE_VERSION

Traits

BufReadText

An extension trait for BufRead which adds functions for reading lines as TextStrings.

ReadText

Add a convenience method for reading Basic Text content.

ReadTextLayered

Extend the ReadLayered trait with read_text_with_status, a method for reading text data.

WriteText

Add a convenience method for reading into TextStr.

Functions

copy_text

Like std::io::copy, but for streams that can operate directly on text strings, so we can avoid re-validating them as text.

copy_text_using_status

Like std::io::copy, but for streams that can operate directly on text strings, so we can avoid re-validating them as text.

default_read_exact_text_substr

Default implementation of ReadText::read_exact_text_substr.

default_read_to_text_string

Default implementation of ReadText::read_to_text_string.

default_write_text_substr

Default implementation of WriteText::write_text_substr.