lightstream 0.4.4

Composable, zero-copy Arrow IPC and native data streaming for Rust with SIMD-aligned I/O, async support, and memory-mapping.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

//! # Asynchronous stdin byte stream
//!
//! Type alias over [`AsyncReadByteStream`] for standard input.
//!
//! ## Use cases
//! - Receive Arrow IPC streams from Unix pipes without loading them fully into memory.
//! - Build CLI tools that compose with other Arrow-aware programs.
//! - Feed piped data directly into async Arrow decoding pipelines.

use crate::enums::BufferChunkSize;
use crate::models::streams::async_read::AsyncReadByteStream;

/// A `Stream` that reads from stdin in fixed-size byte chunks.
pub type StdinByteStream = AsyncReadByteStream<tokio::io::Stdin>;

/// Create a stdin byte stream with the given chunk size.
pub fn from_stdin(size: BufferChunkSize) -> StdinByteStream {
    StdinByteStream::new(tokio::io::stdin(), size)
}

/// Create a stdin byte stream with the default chunk size for pipes.
///
/// Uses `BufferChunkSize::Http` (64 KiB) which works well for most
/// pipe-based streaming scenarios.
pub fn from_stdin_default() -> StdinByteStream {
    from_stdin(BufferChunkSize::Http)
}