Module aws_smithy_http::byte_stream

ByteStream Abstractions

When the SDK returns streaming binary data, the inner Http Body is wrapped in ByteStream. ByteStream provides misuse-resistant primitives to make it easier to handle common patterns with streaming data.

Examples

Writing a ByteStream into a file:

use aws_smithy_http::byte_stream::ByteStream;
use std::error::Error;
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
struct SynthesizeSpeechOutput {
    audio_stream: ByteStream,
}

async fn audio_to_file(
    output: SynthesizeSpeechOutput,
) -> Result<(), Box<dyn Error + Send + Sync>> {
    let mut buf = output.audio_stream.collect().await?;
    let mut file = File::open("audio.mp3").await?;
    file.write_all_buf(&mut buf).await?;
    file.flush().await?;
    Ok(())
}

Converting a ByteStream into Bytes

use bytes::Bytes;
use aws_smithy_http::byte_stream::ByteStream;
use std::error::Error;
struct SynthesizeSpeechOutput {
    audio_stream: ByteStream,
}
async fn load_audio(
    output: SynthesizeSpeechOutput,
) -> Result<Bytes, Box<dyn Error + Send + Sync>> {
    Ok(output.audio_stream.collect().await?.into_bytes())
}

Stream a ByteStream into a file

The previous example is recommended in cases where loading the entire file into memory first is desirable. For extremely large files, you may wish to stream the data directly to the file system, chunk by chunk. This is posible using the futures::Stream implementation.

use bytes::{Buf, Bytes};
use aws_smithy_http::byte_stream::ByteStream;
use std::error::Error;
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
use tokio_stream::StreamExt;
struct SynthesizeSpeechOutput {
    audio_stream: ByteStream,
}

async fn audio_to_file(
    output: SynthesizeSpeechOutput,
) -> Result<(), Box<dyn Error + Send + Sync>> {
    let mut file = File::open("audio.mp3").await?;
    let mut stream = output.audio_stream;
    while let Some(bytes) = stream.next().await {
        let bytes: Bytes = bytes?;
        file.write_all(&bytes).await?;
    }
    file.flush().await?;
    Ok(())
}

Create a ByteStream from a file

Note: This is only available with rt-tokio enabled.

use aws_smithy_http::byte_stream::ByteStream;
use std::path::Path;
struct GetObjectInput {
  body: ByteStream
}

async fn bytestream_from_file() -> GetObjectInput {
    let bytestream = ByteStream::from_path("docs/some-large-file.csv")
        .await
        .expect("valid path");
    GetObjectInput { body: bytestream }
}

If you want more control over how the file is read, such as specifying the size of the buffer used to read the file or the length of the file, use an FsBuilder.

use aws_smithy_http::byte_stream::ByteStream;
use std::path::Path;
struct GetObjectInput {
    body: ByteStream
}

async fn bytestream_from_file() -> GetObjectInput {
    let bytestream = ByteStream::read_from().path("docs/some-large-file.csv")
        .buffer_size(32_784)
        .file_size(123_456)
        .build()
        .await
        .expect("valid path");
    GetObjectInput { body: bytestream }
}

Structs

AggregatedBytes

Non-contiguous Binary Data Storage

ByteStream

Stream of binary data

Error

FsBuilder

Builder for creating ByteStreams from a file/path, with full control over advanced options.