Expand description
OpenDAL is the Open Data Access Layer that connect the whole world together.
Supported Services
Services | Description |
---|---|
azblob | Azure Storage Blob services. |
fs | POSIX alike file system. |
hdfs | Hadoop Distributed File System(HDFS). |
memory | In memory backend support. |
s3 | AWS S3 alike services. |
Optional features
compress
: Enable object decompress read support.retry
: Enable operator retry support.services-hdfs
: Enable hdfs service support.
Example
use anyhow::Result;
use futures::StreamExt;
use opendal::services::fs;
use opendal::Metadata;
use opendal::Object;
use opendal::ObjectMode;
use opendal::ObjectStreamer;
use opendal::Operator;
#[tokio::main]
async fn main() -> Result<()> {
// Init Operator
let op = Operator::new(fs::Backend::build().root("/tmp").finish().await?);
// Create object handler.
let o: Object = op.object("test_file");
// Write data info object;
let _: () = o.write("Hello, World!").await?;
// Read data from object;
let bs: Vec<u8> = o.read().await?;
// Read range from object;
let bs: Vec<u8> = o.range_read(1..=11).await?;
// Get object's Metadata
let meta: Metadata = o.metadata().await?;
let name: &str = meta.name();
let path: &str = meta.path();
let mode: ObjectMode = meta.mode();
let length: u64 = meta.content_length();
let content_md5: Option<String> = meta.content_md5();
// Delete object.
let _: () = o.delete().await?;
// List dir object.
let o: Object = op.object("test_dir/");
let mut obs: ObjectStreamer = o.list().await?;
while let Some(entry) = obs.next().await {
let entry: Object = entry?;
}
Ok(())
}
Modules
Providing IO utils like into_sink
, into_stream
.
Providing specific services support.
Structs
Metadata for accessor, users can use this metadata to get information of underlying backend.
Metadata carries all object metadata.
Handler for all object related operations.
User-facing APIs for object and object streams.
Enums
ObjectMode represents the corresponding object’s mode.
Backends that OpenDAL supports
Traits
Underlying trait of all backends for implementors.
BytesRead represents a reader of bytes.
BytesSink represents a sink of bytes.
BytesStream represents a stream of bytes.
BytesWrite represents a writer of bytes.
Layer is used to intercept the operations on the underlying storage.
ObjectStream represents a stream of object.
Type Definitions
BytesReader is a boxed dyn BytesRead
.
BytesWriter is a boxed dyn BytesWrite
.
ObjectStreamer is a boxed dyn ObjectStream