OpenDAL
Open Data Access Layer: Access data freely, painlessly, and efficiently
You may be looking for:
Services
- azblob: Azure Storage Blob services.
- fs: POSIX alike file system.
- ftp: FTP and FTPS support.
- gcs: Google Cloud Storage Service.
- hdfs: Hadoop Distributed File System(HDFS).
- http: HTTP read-only services.
- ipfs: InterPlanetary File System HTTP Gateway support.
- ipmfs: InterPlanetary File System MFS API support.
- memory: In memory backend.
- obs: Huawei Cloud Object Storage Service (OBS).
- s3: AWS S3 alike services.
Features
Access data freely
- Access different storage services in the same way
- Behavior tests for all services
Access data painlessly
- 100% documents covered
- Powerful
Layers
- Automatic retry support
- Full observability support: logging, tracing, metrics.
- Native decompress support
- Native service-side encryption support
Access data efficiently
- Zero cost: mapping to underlying API calls directly
- Auto metadata reuse: avoid extra
metadata
calls
Quickstart
use Result;
use StreamExt;
use TryStreamExt;
use ObjectEntry;
use ObjectStreamer;
use Object;
use ObjectMetadata;
use ObjectMode;
use Operator;
use Scheme;
async
More examples could be found at Documentation.
Projects
- databend: A modern Elasticity and Performance cloud data warehouse.
Contributing
Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.
Getting help
Submit issues for bug report or asking questions in discussion.