Expand description

OpenDAL is the Open Data Access Layer that connect the whole world together.

Supported Services

ServicesDescription
azblobAzure Storage Blob services.
fsPOSIX alike file system.
hdfsHadoop Distributed File System(HDFS).
memoryIn memory backend support.
s3AWS S3 alike services.

Optional features

  • compress: Enable object decompress read support.
  • retry: Enable operator retry support.
  • services-hdfs: Enable hdfs service support.

Example

use anyhow::Result;
use futures::StreamExt;
use opendal::services::fs;
use opendal::Metadata;
use opendal::Object;
use opendal::ObjectMode;
use opendal::ObjectStreamer;
use opendal::Operator;

#[tokio::main]
async fn main() -> Result<()> {
    // Init Operator
    let op = Operator::new(fs::Backend::build().root("/tmp").finish().await?);

    // Create object handler.
    let o: Object = op.object("test_file");

    // Write data info object;
    let _: () = o.write("Hello, World!").await?;

    // Read data from object;
    let bs: Vec<u8> = o.read().await?;

    // Read range from object;
    let bs: Vec<u8> = o.range_read(1..=11).await?;

    // Get object's Metadata
    let meta: Metadata = o.metadata().await?;
    let name: &str = meta.name();
    let path: &str = meta.path();
    let mode: ObjectMode = meta.mode();
    let length: u64 = meta.content_length();
    let content_md5: Option<String> = meta.content_md5();

    // Delete object.
    let _: () = o.delete().await?;

    // List dir object.
    let o: Object = op.object("test_dir/");
    let mut obs: ObjectStreamer = o.list().await?;
    while let Some(entry) = obs.next().await {
        let entry: Object = entry?;
    }

    Ok(())
}

Modules

Providing IO utils like into_sink, into_stream.

Operations used by Accessor.

Providing specific services support.

Structs

Metadata for accessor, users can use this metadata to get information of underlying backend.

Metadata carries all object metadata.

Handler for all object related operations.

User-facing APIs for object and object streams.

Enums

ObjectMode represents the corresponding object’s mode.

Backends that OpenDAL supports

Traits

Underlying trait of all backends for implementors.

BytesRead represents a reader of bytes.

BytesSink represents a sink of bytes.

BytesStream represents a stream of bytes.

BytesWrite represents a writer of bytes.

Layer is used to intercept the operations on the underlying storage.

ObjectStream represents a stream of object.

Type Definitions

BytesReader is a boxed dyn BytesRead.

BytesWriter is a boxed dyn BytesWrite.

ObjectStreamer is a boxed dyn ObjectStream