Skip to main content

Module pipe

Module pipe 

Source
Available on crate feature pipe only.
Expand description

Convert a stream (usually async) of bytes::Bytes chunks into JSON lexical tokens.

The Bytes chunks can be produced either using the asynchronous programming model or using a multi-threaded programming model.

§Difference between pipe and read

Both this module and the read module provide lexical analyzers that scan JSON read from an external source.

For the read module, that external source is a std::io::Read. A consequence of its design is that read::ReadAnalyzer has to read from the Read into its internal buffers, so every byte of input has to be copied or moved in order to be scanned by the lexical analyzer.

In contrast, the external source for this module is a Pipe that provides input chunks to the PipeAnalyzer as Bytes buffers. Bytes buffers are reference-counted, immutable values that support shared ownership. Because of these features, input bytes already resident in memory can be sent to a PipeAnalyzer without any copying or allocation. These properties make PipeAnalyzer an excellent fit for some use cases, like web programming, where chunks of the JSON text are already in memory because they were read by some other subsystem, such as the network stack.

Structs§

Content
Text content of a JSON token identified by a PipeAnalyzer.
Error
Lexical analysis error detected by a PipeAnalyzer.
Literal
Zero allocation view of the literal text content of a JSON token.
LiteralBuf
A Buf implementation for Literal.
PipeAnalyzer
A lexical::Analyzer to tokenize JSON text from a stream of Bytes buffers.

Traits§

Pipe
Provides JSON text to a PipeAnalyzer as a stream of bytes::Bytes buffers.