Crate paging_stream

A utility to simplify consuming paginated data sources as a futures::Stream.

This crate provides the Paginated trait, which you implement for your API client or repository, and the PagingStream struct, which wraps your type and yields items as a stream. This allows consumers of your API to work with a continuous stream of data, abstracting away the underlying pagination logic.

Example

use futures::{StreamExt, future::Future};
use paging_stream::{Paginated, PagingStream};

// 1. Define a client/repository struct.
struct MyApiClient;

// 2. Define your types for parameters, items, and errors.
struct MyParams {
    since: usize,
    until: usize,
    limit: usize
}

// 3. Implement the `Paginated` trait for your client.
impl Paginated for MyApiClient {
    type Params = MyParams;
    type Item = usize;
    type Error = ();

    fn fetch_page(
        &self,
        params: Self::Params,
    ) -> impl Future<Output = Result<(Vec<Self::Item>, Option<Self::Params>), Self::Error>>
    + Send
    + 'static {
        async move {
            // Replace with your actual asynchronous data fetching logic.
            //
            // - `params`: Contains the necessary information to fetch the current page.
            // - Return `Ok((items, next_params))` where:
            //   - `items`: A `Vec` of fetched items for the current page.
            //   - `next_params`: An `Option<Self::Params>`:
            //     - `Some(params)`: Contains the parameters needed to fetch the *next* page.
            //     - `None`: Signifies that there are no more pages.
            // - Return `Err(your_error)` if fetching fails.
           Ok((Vec::new(), None)) // Placeholder for example
        }
    }
}

async fn consume_as_stream() {
    let client = MyApiClient;
    let initial_params = MyParams {
        since: 0,
        until: 100,
        limit: 20
    };

    // 4. Create a `PagingStream`.
    let mut stream = PagingStream::new(client, initial_params);

    // 5. Consume the stream.
    while let Some(result) = stream.next().await {
        match result {
            Ok(item) => { /* process `item` */ }
            Err(e) => { /* handle `e` */ break; }
        }
    }
}

Structs

PagingStream

A stream that wraps a Paginated type to provide continuous, lazy-loaded data.

Traits

Paginated

Represents a data source that can be paginated.