Struct ClientBuilder 

pub struct ClientBuilder { /* private fields */ }

Builds a new Client instance. Configs will be loaded with the following precedence:

• If method ClientBuilder::with_config_dir is invoked, configs will be loaded from ${config_dir}/{core,hdfs}-site.xml
• If the HADOOP_CONF_DIR environment variable is defined, configs will be loaded from ${HADOOP_CONF_DIR}/{core,hdfs}-site.xml
• If the HADOOP_HOME environment variable is defined, configs will be loaded from ${HADOOP_HOME}/etc/hadoop/{core,hdfs}-site.xml
• Otherwise no configs are defined

Finally, configs set by with_config will override the configs loaded above.

If no URL is defined, the fs.defaultFS config must be defined and is used as the URL.

Examples

Create a new client with given config directory

let client = ClientBuilder::new()
    .with_config_dir("/opt/hadoop/etc/hadoop")
    .build()
    .unwrap();

Create a client that acquires credentials directly from a keytab:

An explicit principal is required. A keytab without a supplied cache uses a native MEMORY: credential cache.

let client = ClientBuilder::new()
    .with_url("hdfs://namenode.example.com:9000")
    .with_kerberos_principal("client@EXAMPLE.COM")
    .with_kerberos_keytab("/run/secrets/client.keytab")
    .build()
    .unwrap();

Create a new client with the environment variable

unsafe { std::env::set_var("HADOOP_CONF_DIR", "/opt/hadoop/etc/hadoop") };
let client = ClientBuilder::new()
    .build()
    .unwrap();

Create a new client using the fs.defaultFS config

let client = ClientBuilder::new()
    .with_config(vec![("fs.defaultFS", "hdfs://127.0.0.1:9000")])
    .build()
    .unwrap();

Create a new client connecting to a specific URL:

let client = ClientBuilder::new()
    .with_url("hdfs://127.0.0.1:9000")
    .build()
    .unwrap();

Create a new client using a dedicated tokio runtime for spawned tasks and IO operations

let client = ClientBuilder::new()
    .with_url("hdfs://127.0.0.1:9000")
    .with_io_runtime(tokio::runtime::Runtime::new().unwrap())
    .build()
    .unwrap();

Create a client with an explicit Kerberos credential cache:

let client = ClientBuilder::new()
    .with_url("hdfs://namenode.example.com:9000")
    .with_kerberos_principal("client@EXAMPLE.COM")
    .with_kerberos_cache("FILE:/run/krb5/client.ccache")
    .build()
    .unwrap();

Implementations

impl ClientBuilder

pub fn new() -> Self

Create a new ClientBuilder

pub fn with_url(self, url: impl Into<String>) -> Self

Set the URL to connect to. Can be the address of a single NameNode, or a logical NameService

pub fn with_config( self, config: impl IntoIterator<Item = (impl Into<String>, impl Into<String>)>, ) -> Self

Set configs to use for the client. The provided configs will override any found in the config files loaded

pub fn with_config_dir(self, config_dir: impl Into<String>) -> Self

Set the configration directory path to read from. The provided path will override the one provided by environment variable.

pub fn with_io_runtime(self, runtime: impl Into<IORuntime>) -> Self

Use a dedicated tokio runtime for spawned tasks and IO operations. Can either take ownership of a whole Runtime or take a Handle to an externally owned runtime.

pub fn with_user(self, user: impl Into<String>) -> Self

Set the effective user for the client. If not set, the client will detect user from environment variables HADOOP_USER_NAME or HADOOP_PROXY_USER.

pub fn with_kerberos_principal(self, principal: impl Into<String>) -> Self

Set the Kerberos principal used by this client.

pub fn with_kerberos_keytab(self, keytab: impl Into<String>) -> Self

Set the Kerberos keytab used by this client.

pub fn with_kerberos_cache(self, cache: impl Into<String>) -> Self

Set the Kerberos credential cache used by this client.

pub fn build(self) -> Result<Client>

Create the Client instance from the provided settings

Trait Implementations

impl Default for ClientBuilder

fn default() -> ClientBuilder

Returns the “default value” for a type.