Crate remotefs_gcs 

remotefs-gcs

An asynchronous remotefs::AsyncRemoteFs client backed by Google Cloud Storage. Every path is absolute and rooted at the bucket (/ is the bucket root); trailing-slash marker objects and implicit object prefixes are presented as directories.

Installation

[dependencies]
remotefs = "1"
remotefs-gcs = "1"
futures = "0.3"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Feature	Default	Purpose
find	yes	Enables remotefs::find_async
no-log	no	Disables logging from this crate
tokio	no	Enables BlockingGoogleCloudStorageFs for blocking callers
with-containers	no	Enables the local testbench integration suite
with-gcs-ci	no	Enables the optional live smoke test

Application Default Credentials

GoogleCloudStorageFs::new uses Application Default Credentials (ADC):

use std::path::Path;

use remotefs::AsyncRemoteFs;
use remotefs::fs::{ReadOptions, WriteOptions};
use remotefs_gcs::GoogleCloudStorageFs;

let mut client = GoogleCloudStorageFs::new("my-bucket");
client.connect().await?;
let mut source = futures::io::Cursor::new(b"hello".to_vec());
client
    .write_file(
        Path::new("/docs/hello.txt"),
        &WriteOptions::default().size_hint(5),
        &mut source,
    )
    .await?;
let mut destination = futures::io::Cursor::new(Vec::new());
client
    .read_file(
        Path::new("/docs/hello.txt"),
        &ReadOptions::default().offset(1).length(3),
        &mut destination,
    )
    .await?;
assert_eq!(destination.into_inner(), b"ell");
client.disconnect().await?;

Explicit credentials

Use GoogleCloudStorageCredentials::custom to pass any credentials provider supported by google-cloud-auth:

use remotefs::AsyncRemoteFs;
use remotefs_gcs::credentials::Builder;
use remotefs_gcs::{GoogleCloudStorageCredentials, GoogleCloudStorageFs};

let credentials = Builder::default().build()?;
let mut client = GoogleCloudStorageFs::with_credentials(
    "my-bucket",
    GoogleCloudStorageCredentials::custom(credentials),
);
client.connect().await?;
client.disconnect().await?;

GoogleCloudStorageCredentials::anonymous is useful for public buckets and local emulators. Pair it with GoogleCloudStorageFs::endpoint when the service is not running at Google’s default endpoint, and GoogleCloudStorageFs::control_endpoint for emulators with separate object-data and metadata endpoints.

Blocking usage

Enable the tokio feature and call GoogleCloudStorageFs::into_blocking to get a BlockingGoogleCloudStorageFs, which implements remotefs::RemoteFs and can be stored as Box<dyn RemoteFs>. It must not be called from inside an async context. The supplied handle must belong to a multi-thread Tokio runtime.

use remotefs::RemoteFs;
use remotefs_gcs::GoogleCloudStorageFs;

let runtime = tokio::runtime::Runtime::new()?;
let mut client: Box<dyn RemoteFs> = Box::new(
    GoogleCloudStorageFs::new("my-bucket").into_blocking(runtime.handle().clone()),
);
client.connect()?;
client.disconnect()?;

Filesystem semantics

The root directory / always exists and is never an object. list_dir consumes every page and merges objects with prefixes. remove_dir_all removes every object below its path with one flat listing.

open returns an owned read stream over a ranged ReadObject request; offsets and lengths are honored natively (Capabilities::RANGE_READ). create returns an owned write stream that feeds a resumable upload; the object is committed only when finish succeeds, and a dropped stream creates nothing. copy uses rewrite requests until completion; rename copies and then deletes the source and is not atomic. append, set_metadata, symlink, and exec return remotefs::RemoteErrorType::UnsupportedFeature.

Modules

backoff_policy

Re-exports Google Cloud SDK backoff policy types.

credentials

Configures authentication for Google Cloud Storage.

retry_policy

Client retry policy

retry_throttler

Re-exports Google Cloud SDK retry-throttling types.

Structs

GoogleCloudStorageFs

An asynchronous Google Cloud Storage filesystem client.

Enums

GoogleCloudStorageCredentials

Selects credentials used to authenticate storage requests.