faucet_stream/lib.rs
1//! # faucet-stream
2//!
3//! A declarative, config-driven REST API client for Rust with pluggable
4//! authentication, pagination, record transforms, schema inference,
5//! and incremental replication.
6//!
7//! ## Overview
8//!
9//! Configure a [`RestStreamConfig`] once, call [`RestStream::fetch_all()`] or
10//! [`RestStream::fetch_all_as::<T>()`], and get all records — no manual
11//! pagination loop, no auth boilerplate.
12//!
13//! ## Key capabilities
14//!
15//! - **Authentication**: Bearer, Basic, API Key (header or query), OAuth2
16//! (client credentials with automatic token caching), or custom headers
17//! - **Pagination**: cursor/token, page number, offset/limit, Link header,
18//! or next-link-in-body — all with automatic loop detection
19//! - **JSONPath extraction**: point at where records live in any JSON response
20//! - **Record transforms**: flatten nested objects, regex key renaming,
21//! snake_case normalisation, or custom closures (feature-gated)
22//! - **Schema inference**: derive a JSON Schema from sampled records via
23//! [`RestStream::infer_schema()`]
24//! - **Incremental replication**: bookmark-based filtering via
25//! [`RestStream::fetch_all_incremental()`]
26//! - **Partitions**: run the same config across multiple contexts
27//! (e.g. per-org, per-repo) with path placeholder substitution
28//! - **Retries with backoff**: exponential backoff with 429 rate-limit handling
29//! - **Streaming**: process pages as they arrive with [`RestStream::stream_pages()`]
30
31pub mod auth;
32pub mod config;
33pub mod error;
34pub mod extract;
35pub mod pagination;
36pub mod replication;
37pub mod retry;
38pub mod schema;
39pub mod stream;
40pub mod transform;
41
42pub use auth::oauth2::DEFAULT_EXPIRY_RATIO;
43pub use auth::{Auth, fetch_oauth2_token};
44pub use config::RestStreamConfig;
45pub use error::FaucetError;
46pub use pagination::PaginationStyle;
47pub use replication::ReplicationMethod;
48pub use stream::RestStream;
49pub use transform::RecordTransform;