Crate oxigdal_cloud 

Advanced cloud storage backends for OxiGDAL

This crate provides comprehensive cloud storage integration for OxiGDAL, including:

• Cloud Providers: S3, Azure Blob Storage, Google Cloud Storage
• Authentication: OAuth 2.0, service accounts, API keys, SAS tokens, IAM roles
• Advanced Caching: Multi-level cache with memory and disk tiers, compression, LRU+LFU eviction
• Intelligent Prefetching: Predictive prefetch, access pattern analysis, bandwidth management
• Retry Logic: Exponential backoff, jitter, circuit breaker, retry budgets
• HTTP Backend: Enhanced HTTP/HTTPS with authentication and retry support

Features

• s3 - AWS S3 support
• azure-blob - Azure Blob Storage support
• gcs - Google Cloud Storage support
• http - HTTP/HTTPS backend
• cache - Advanced caching layer
• prefetch - Intelligent prefetching
• retry - Retry logic with backoff

Examples

AWS S3

use oxigdal_cloud::backends::S3Backend;
use oxigdal_cloud::backends::CloudStorageBackend;

let backend = S3Backend::new("my-bucket", "data/zarr")
    .with_region("us-west-2");

// Get object
let data = backend.get("file.tif").await?;

// Put object
backend.put("output.tif", &data).await?;

Multi-cloud Abstraction

use oxigdal_cloud::CloudBackend;

// Parse URL and create appropriate backend
let backend = CloudBackend::from_url("s3://bucket/file.tif")?;
let data = backend.get().await?;

Advanced Caching

use oxigdal_cloud::cache::{CacheConfig, MultiLevelCache};
use bytes::Bytes;

let config = CacheConfig::new()
    .with_max_memory_size(100 * 1024 * 1024) // 100 MB
    .with_cache_dir("/tmp/oxigdal-cache");

let cache = MultiLevelCache::new(config)?;

// Cache data
cache.put("key".to_string(), Bytes::from("data")).await?;

// Retrieve from cache
let data = cache.get(&"key".to_string()).await?;

Re-exports

pub use error::CloudError;

pub use error::Result;

pub use backends::s3::S3Backend;

pub use backends::http::HttpBackend;

pub use multicloud::CloudProvider;

pub use multicloud::CloudProviderConfig;

pub use multicloud::CloudRegion;

pub use multicloud::CrossCloudTransferConfig;

pub use multicloud::CrossCloudTransferResult;

pub use multicloud::MultiCloudManager;

pub use multicloud::MultiCloudManagerBuilder;

pub use multicloud::ProviderHealth;

pub use multicloud::RoutingStrategy;

pub use multicloud::TransferCostEstimate;

Modules

auth

Authentication strategies for cloud storage backends

backends

Cloud storage backend implementations

cache

Advanced multi-level caching layer for cloud storage

error

Error types for cloud storage operations

multicloud

Multi-cloud support with unified interface, failover, and intelligent routing

retry

Retry logic with exponential backoff and circuit breaker pattern

Enums

CloudBackend

Multi-cloud storage backend abstraction