Crate opendal

source ·
Expand description

OpenDAL is the Open Data Access Layer to freely, painlessly, and efficiently access data.


Service represents a backend scheme that OpenDAL supported.

OpenDAL supports the following services now:

azblobAzure Storage Blob services.
azdfsAzure Data Lake Storage Gen2 services.
fsPOSIX alike file system.
ftpFTP and FTPS support.
gcsGoogle Cloud Storage service.
ghacGithub Action Cache service.
hdfsHadoop Distributed File System(HDFS).
httpHTTP read-only backend.
ipfsIPFS HTTP Gateway support.
ipmfsIPFS Mutable File System support.
memcachedMemcached serivce.
memoryIn memory backend support.
mokamoka backend support.
obsHuawei Cloud OBS service.
ossAliyun Object Storage Service (OSS).
redisRedis service.
rocksdbRocksDB service.
s3AWS S3 alike services.
webdavWebDAV services.

More services support is tracked at opendal#5


Layer is the mechanism to intercept operations.

OpenDAL supports the following layers now:

ConcurrentLimitLayerConcurrent request limit.
CacheLayerCache supports.
ImmutableIndexLayerImmutable in-memory index.
LoggingLayerLogging for every operations.
MetricsLayerMetrics for every operations.
RetryLayerRetry for failed operations.
SubdirLayerAllow switching directory.
TracingLayerTracing for every operations.

Optional features


  • layers-all: Enable all layers support.
  • layers-metrics: Enable operator metrics support.
  • layers-tracing: Enable operator tracing support.


  • services-ftp: Enable ftp service support.
  • services-hdfs: Enable hdfs service support.
  • services-moka: Enable moka service support.
  • services-ipfs: Enable ipfs service support.
  • services-redis: Enable redis service support.
  • services-rocksdb: Enable rocksdb service support.

Dependencies features

  • compress: Enable object decompress read support.
  • rustls: Enable TLS functionality provided by rustls, enabled by default
  • native-tls: Enable TLS functionality provided by native-tls
  • native-tls-vendored: Enable the vendored feature of native-tls


More examples could be found at

use anyhow::Result;
use backon::ExponentialBackoff;
use futures::StreamExt;
use futures::TryStreamExt;
use opendal::layers::LoggingLayer;
use opendal::layers::RetryLayer;
use opendal::Object;
use opendal::ObjectMetadata;
use opendal::ObjectMode;
use opendal::Operator;
use opendal::Scheme;

async fn main() -> Result<()> {
    // Init a fs operator
    let op = Operator::from_env(Scheme::Fs)?
        // Init with logging layer enabled.
        // Init with retry layer enabled.

    // Create object handler.
    let o = op.object("test_file");

    // Write data
    o.write("Hello, World!").await?;

    // Read data
    let bs =;

    // Fetch metadata
    let name =;
    let path = o.path();
    let meta = o.metadata().await?;
    let mode = meta.mode();
    let length = meta.content_length();

    // Delete

    // Readdir
    let o = op.object("test_dir/");
    let mut ds = o.list().await?;
    while let Some(entry) = ds.try_next().await? {
        let path = entry.path();
        let mode = entry.mode().await?;



Providing Layer trait and its implementations.
Raw modules provide raw APIs that used by underlying services
Providing specific services support.


BatchOperator is used to take batch operations like walk_dir and remove_all, should be constructed by Operator::batch().
Error is the error struct returned by all opendal functions.
Object is the handler for all object related operations.
ObjectLister is returned by Object::list to list objects.
Metadata carries all object metadata.
ObjectMultipart represent an ongoing multipart upload.
ObjectPart is generated by write_multipart operation, carries required information for complete_multipart.
ObjectReader is the public API for users.
Args for abort_multipart operation.
Args for complete_multipart operation.
Args for create operation.
Args for create_multipart operation.
Args for delete operation.
Args for list operation.
Args for presign operation.
Args for read operation.
Args for stat operation.
Args for write operation.
Args for write_multipart operation.
User-facing APIs for object and object streams.
Metadata for operator, users can use this metadata to get information of operator.


ErrorKind is all kinds of opendal’s Error.
ObjectMode represents the corresponding object’s mode.
Presign operation used for presign.
Services that OpenDAL supports


Layer is used to intercept the operations on the underlying storage.

Type Definitions

Result that is a wrapper of Reustl<T, opendal::Error>