Expand description

OpenDAL is the Open Data Access Layer that connect the whole world together.

Supported Services

ServicesDescription
azblobAzure Storage Blob services.
fsPOSIX alike file system.
hdfsHadoop Distributed File System(HDFS).
httpHTTP read-only backend.
memoryIn memory backend support.
s3AWS S3 alike services.

Optional features

  • compress: Enable object decompress read support.
  • retry: Enable operator retry support.
  • services-hdfs: Enable hdfs service support.
  • services-http: Enable http service support.
  • rustls: Use rustls instead openssl for https connection
  • serde: Implement serde::{Serialize,Deserialize} for ObjectMetadata.

Example

use anyhow::Result;
use futures::StreamExt;
use futures::TryStreamExt;
use opendal::DirEntry;
use opendal::DirStreamer;
use opendal::Object;
use opendal::ObjectMetadata;
use opendal::ObjectMode;
use opendal::Operator;
use opendal::Scheme;

#[tokio::main]
async fn main() -> Result<()> {
    // Init Operator
    let op = Operator::from_env(Scheme::Fs)?;

    // Create object handler.
    let o = op.object("test_file");

    // Write data info object;
    o.write("Hello, World!").await?;

    // Read data from object;
    let bs = o.read().await?;

    // Read range from object;
    let bs = o.range_read(1..=11).await?;

    // Get object's path
    let name = o.name();
    let path = o.path();

    // Fetch more meta about object.
    let meta = o.metadata().await?;
    let mode = meta.mode();
    let length = meta.content_length();
    let content_md5 = meta.content_md5();
    let etag = meta.etag();

    // Delete object.
    o.delete().await?;

    // List dir object.
    let o = op.object("test_dir/");
    let mut ds = o.list().await?;
    while let Some(entry) = ds.try_next().await? {
        let path = entry.path();
        let mode = entry.mode();
    }

    Ok(())
}

Modules

Providing IO utils like into_sink, into_stream.

Operations used by Accessor.

Providing specific services support.

Structs

Metadata for accessor, users can use this metadata to get information of underlying backend.

BatchOperator is used to take batch operations like walk_dir and remove_all, should be constructed by Operator::batch().

DirEntry is returned by DirStream during object list.

Handler for all object related operations.

Metadata carries all object metadata.

User-facing APIs for object and object streams.

Enums

ObjectMode represents the corresponding object’s mode.

Backends that OpenDAL supports

Traits

Underlying trait of all backends for implementors.

BytesRead represents a reader of bytes.

BytesSink represents a sink of bytes.

BytesStream represents a stream of bytes.

BytesWrite represents a writer of bytes.

DirStream represents a stream of Dir.

Layer is used to intercept the operations on the underlying storage.

Type Definitions

BytesReader is a boxed dyn BytesRead.

BytesWriter is a boxed dyn BytesWrite.

DirStreamer is a boxed dyn DirStream