Expand description
OpenDAL is the Open Data Access Layer that connect the whole world together.
Supported Services
Services | Description |
---|---|
azblob | Azure Storage Blob services. |
fs | POSIX alike file system. |
ftp | FTP and FTPS support. |
gcs | Google Cloud Storage service. |
hdfs | Hadoop Distributed File System(HDFS). |
http | HTTP read-only backend. |
ipfs | IPFS HTTP Gateway support. |
ipmfs | IPFS Mutable File System support. |
memory | In memory backend support. |
obs | Huawei Cloud OBS service. |
s3 | AWS S3 alike services. |
Optional features
Layers
layers-all
: Enable all layers support.layers-retry
: Enable operator retry support.layers-metrics
: Enable operator metrics support.layers-tracing
: Enable operator tracing support.
Services
services-ftp
: Enable ftp service support.services-hdfs
: Enable hdfs service support.services-ipfs
: Enable ipfs service support.
Dependencies features
compress
: Enable object decompress read support.rustls
: Use rustls instead openssl for https connectionserde
: Implement serde::{Serialize,Deserialize} for ObjectMetadata.
Example
use anyhow::Result;
use futures::StreamExt;
use futures::TryStreamExt;
use opendal::DirEntry;
use opendal::DirStreamer;
use opendal::Object;
use opendal::ObjectMetadata;
use opendal::ObjectMode;
use opendal::Operator;
use opendal::Scheme;
#[tokio::main]
async fn main() -> Result<()> {
// Init Operator
let op = Operator::from_env(Scheme::Fs)?;
// Create object handler.
let o = op.object("test_file");
// Write data info object;
o.write("Hello, World!").await?;
// Read data from object;
let bs = o.read().await?;
// Read range from object;
let bs = o.range_read(1..=11).await?;
// Get object's path
let name = o.name();
let path = o.path();
// Fetch more meta about object.
let meta = o.metadata().await?;
let mode = meta.mode();
let length = meta.content_length();
let content_md5 = meta.content_md5();
let etag = meta.etag();
// Delete object.
o.delete().await?;
// List dir object.
let o = op.object("test_dir/");
let mut ds = o.list().await?;
while let Some(entry) = ds.try_next().await? {
let path = entry.path();
let mode = entry.mode();
}
Ok(())
}
Modules
Providing IO utils like
into_sink
, into_stream
.Providing Layer trait and its implementations.
Providing specific services support.
Structs
Metadata for accessor, users can use this metadata to get information of underlying backend.
BatchOperator is used to take batch operations like walk_dir and remove_all, should
be constructed by
Operator::batch()
.Handler for all object related operations.
Metadata carries all object metadata.
ObjectMultipart represent an ongoing multipart upload.
ObjectPart is generated by
write_multipart
operation, carries
required information for complete_multipart
.User-facing APIs for object and object streams.
Enums
ObjectMode represents the corresponding object’s mode.
Services that OpenDAL supports
Traits
Underlying trait of all backends for implementors.
BlockingBytesRead represents a blocking reader of bytes.
BytesRead represents a reader of bytes.
BytesSink represents a sink of bytes.
BytesStream represents a stream of bytes.
BytesWrite represents a writer of bytes.
DirIterate represents an iterator of Dir.
DirStream represents a stream of Dir.
Layer is used to intercept the operations on the underlying storage.
Type Definitions
BlockingBytesReader is a boxed dyn
BlockingBytesRead
.BytesReader is a boxed dyn
BytesRead
.BytesWriter is a boxed dyn
BytesWrite
.DirIterator is a boxed dyn
DirIterate
DirStreamer is a boxed dyn
DirStream