Crate reqwest_streams

Streaming responses support for reqwest for different formats:

• JSON array stream format
• JSON Lines (NL/NewLines) format
• CSV stream format
• Protobuf len-prefixed stream format
• Apache Arrow IPC stream format

This type of responses are useful when you are reading huge stream of objects from some source (such as database, file, etc) and want to avoid huge memory allocations to store on the server side.

Features

Note: The default features do not include any formats.

• json: JSON array and JSON Lines (JSONL) stream formats
• csv: CSV stream format
• protobuf: Protobuf len-prefixed stream format
• arrow: Apache Arrow IPC stream format

Example

use futures::stream::BoxStream as _;
use reqwest_streams::JsonStreamResponse as _;
use serde::Deserialize;

#[derive(Debug, Clone, Deserialize)]
struct MyTestStructure {
    some_test_field: String
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {

    let _stream = reqwest::get("http://localhost:8080/json-array")
        .await?
        .json_array_stream::<MyTestStructure>(1024);

    Ok(())
}

More and complete examples available on the github in the examples directory.

Need server support?

There is the same functionality:

• axum-streams.

Modules

error

Error types for streaming responses.

Traits

ArrowIpcStreamResponsearrow

Extension trait for reqwest::Response that provides streaming support for the Apache Arrow IPC format.

CsvStreamResponsecsv

Extension trait for reqwest::Response that provides streaming support for the CSV format.

JsonStreamResponsejson

Extension trait for reqwest::Response that provides streaming support for the JSON array and JSON Lines (NL/NewLines) formats.

ProtobufStreamResponseprotobuf

Extension trait for reqwest::Response that provides streaming support for the Protobuf format.

Type Aliases

StreamBodyResult

Alias for the Result type returned by streaming responses.