Crate oxigdal_streaming 

Real-time data processing and streaming pipelines for OxiGDAL.

This crate provides a comprehensive streaming framework for processing geospatial data in real-time. It includes:

• Stream traits and abstractions with backpressure handling
• Windowing and watermarking for event-time processing
• Rich set of transformations (map, filter, join, etc.)
• Stateful operations with checkpointing
• State backends (RocksDB) for fault tolerance

Example

use oxigdal_streaming::core::Stream;

// Create a stream and apply transformations
// let stream = Stream::new();
// let transformed = stream.map(|x| x * 2).filter(|x| x > 10);

Re-exports

pub use error::Result;

pub use error::StreamingError;

Modules

arrow_ipc

Zero-copy Arrow IPC framing for inter-process communication.

cloud

Cloud-native I/O abstractions for OxiGDAL streaming.

core

Core streaming abstractions and traits.

error

Error types for the streaming module.

io_coalescing

Parallel I/O coalescing for cloud object storage reads.

metrics

Metrics collection and monitoring for streaming operations.

mmap

Memory-mapped file support for local reads.

state

State management for stateful stream processing.

transformations

Stream transformations and operations.

v2

Streaming v2: production-quality additions to the OxiGDAL streaming framework.

windowing

Windowing and watermarking for event-time processing.