This repository contains a project to define a text format called Basic Text, a subset of Unicode being developed to focus on text: it excludes unprintable control characters, characters which depend on out-of-band information to interpret, non-characters, deprecated characters, and more, while aiming to preserve everything of practical use to plain text and formats built on top of plain text. See the Background document for further background information.
It also includes a Rust crate which aims to implement the Basic Text format, providing several utilities:
-
TextStringandTextStrare similar to the standard library'sStringandstr, but use the Basic Text string format. -
TextReaderandTextWriterare input and output streams which use the Basic Text stream format. On input, content is converted in a way which is lossy with respect to the original bytestream. Output uses the "strict" conversion method, in which invalid content is diagnosed with errors. -
TextDuplexeris aDuplexfor reading and writing on an interactive stream using Basic Text.
The code here is usable, but not very mature or optimized yet. It implements
most of the Basic Text spec, though see the TODOs in the tests directory for
remaining missing pieces.