Crate hdrs

source ·
Expand description

hdrs is a HDFS Native Client in Rust based on hdfs-sys.


use std::io::{Read, Write};

use hdrs::Client;
use hdrs::ClientBuilder;
let fs = ClientBuilder::new("default").connect()?;

let mut f = fs
let n = f.write("Hello, World!".as_bytes())?;

let mut f = fs.open_file().read(true).open("/tmp/hello.txt")?;
let mut buf = vec![0; 1024];
let n = buf)?;

let _ = fs.remove_file("/tmp/hello.txt")?;


  • async_file: Enable async operation support
  • vendored: Ignore lib loading logic, enforce to complie and staticly link libhdfs


hdrs depends on hdfs-sys which links libjvm to work.

Please make sure JAVA_HOME is set correctly:

export JAVA_HOME=/path/to/java
  • Enable vendored feature to compile libhdfs and link in static.
  • Specify HDFS_LIB_DIR or HADOOP_HOME to load from specified path instead of compile.
  • Specify HDFS_STATIC=1 to link libhdfs in static.
  • And finally, we will fallback to compile libhdfs and link in static.


hdrs depends on hdfs-sys which uses JNI to call functions provided by jars that provided by hadoop releases.

Please also make sure HADOOP_HOME, LD_LIBRARY_PATH, CLASSPATH is set correctly during runtime:

export HADOOP_HOME=/path/to/hadoop
export CLASSPATH=$(${HADOOP_HOME}/bin/hadoop classpath --glob)

If libhdfs is configued to link dynamiclly, please also add ${HADOOP_HOME}/lib/native in LD_LIBRARY_PATH to make sure linker can find

export LD_LIBRARY_PATH=${JAVA_HOME}/lib/server:${HADOOP_HOME}/lib/native:${LD_LIBRARY_PATH}


  • Async version of file.
  • Client holds the underlying connection to hdfs clusters.
  • The builder of connecting to hdfs clusters.
  • File will hold the underlying pointer to hdfsFile.
  • Metadata of a path.
  • Options and flags which can be used to configure how a file is opened.
  • Readdir is an Iterator of metadata, returned by read_dir