Modules

Structs

hdfsEncryptionFileInfo - Information about an encryption file/directory.

hdfsEncryptionZoneInfo- Information about an encryption zone.

hdfsFileInfo - Information about a file/directory.

Enums

Functions

hdfsAvailable - Number of bytes that can be read from this input stream without blocking. @param fs The configured filesystem handle. @param file The file handle. @return Returns available bytes; -1 on error.

Set a configuration string for an HdfsBuilder.

Connect to HDFS using the parameters defined by the builder.

Do nothing, we always create a new instance

Set the path to the Kerberos ticket cache to use when connecting to the HDFS cluster.

Set the HDFS NameNode to connect to.

Set the port of the HDFS NameNode to connect to.

Set the token used to authenticate

Set the username to use when connecting to the HDFS cluster.

Cancel a delegation token.

hdfsChmod @param fs The configured filesystem handle. @param path the path to the file or directory @param mode the bitmask to set it to @return 0 on success else -1

Change the user and/or group of a file or directory.

hdfsCloseFile - Close an open file. @param fs The configured filesystem handle. @param file The file handle. @return Returns 0 on success, -1 on error. On error, errno will be set appropriately. If the hdfs file was valid, the memory associated with it will be freed at the end of this call, even if there was an I/O error.

Get a configuration integer.

Get a configuration string.

Free a configuration string found with hdfsConfGetStr.

hdfsConnect - Connect to a hdfs file system. Connect to the hdfs. @param nn The NameNode. See hdfsBuilderSetNameNode for details. @param port The port on which the server is listening. @return Returns a handle to the filesystem or NULL on error. @deprecated Use hdfsBuilderConnect instead.

hdfsConnectAsUser - Connect to a hdfs file system as a specific user Connect to the hdfs. @param nn The NameNode. See hdfsBuilderSetNameNode for details. @param port The port on which the server is listening. @param user the user name (this is hadoop domain user). Or NULL is equivelant to hhdfsConnect(host, port) @return Returns a handle to the filesystem or NULL on error. @deprecated Use hdfsBuilderConnect instead.

hdfsConnect - Connect to an hdfs file system.

hdfsConnect - Connect to an hdfs file system.

hdfsCopy - Copy file from one filesystem to another. @param srcFS The handle to source filesystem. @param src The path of source file. @param dstFS The handle to destination filesystem. @param dst The path of destination file. @return Returns 0 on success, -1 on error.

hdfsCreateDirectory - Make the given file and all non-existent parents into directories. @param fs The configured filesystem handle. @param path The path of the directory. @return Returns 0 on success, -1 on error.

Create encryption zone for the directory with specific key name @param fs The configured filesystem handle. @param path The path of the directory. @param keyname The key name of the encryption zone @return Returns 0 on success, -1 on error.

hdfsDelete - Delete file. @param fs The configured filesystem handle. @param path The path of the file. @param recursive if path is a directory and set to non-zero, the directory is deleted else throws an exception. In case of a file the recursive argument is irrelevant. @return Returns 0 on success, -1 on error.

hdfsDisconnect - Disconnect from the hdfs file system. Disconnect from hdfs. @param fs The configured filesystem handle. @return Returns 0 on success, -1 on error. Even if there is an error, the resources associated with the hdfsFS will be freed.

hdfsExists - Checks if a given path exsits on the filesystem @param fs The configured filesystem handle. @param path The path to look for @return Returns 0 on success, -1 on error.

Determine if a file is open for read.

Determine if a file is open for write.

hdfsWrite - Flush the data. @param fs The configured filesystem handle. @param file The file handle. @return Returns 0 on success, -1 on error.

Free an HDFS builder.

Free a delegation token.

hdfsFreeEncryptionZoneInfo - Free up the hdfsEncryptionZoneInfo array (including fields) @param infos The array of dynamically-allocated hdfsEncryptionZoneInfo objects. @param numEntries The size of the array.

Free the BlockLocation array returned by hdfsGetFileBlockLocations

hdfsFreeFileInfo - Free up the hdfsFileInfo array (including fields) @param infos The array of dynamically-allocated hdfsFileInfo objects. @param numEntries The size of the array.

hdfsFreeHosts - Free up the structure returned by hdfsGetHosts @param hdfsFileInfo The array of dynamically-allocated hdfsFileInfo objects. @param numEntries The size of the array.

Free the array returned by hdfsGetConfiguredNamenodes()

hdfsGetCapacity - Return the raw capacity of the filesystem. @param fs The configured filesystem handle. @return Returns the raw-capacity; -1 on error.

hdfsGetDefaultBlockSize - Get the default blocksize.

Get a delegation token from namenode. The token should be freed using hdfsFreeDelegationToken after canceling the token or token expired.

hdfsEncryptionZoneInfo - Get information about a path as a (dynamically allocated) single hdfsEncryptionZoneInfo struct. hdfsEncryptionZoneInfo should be called when the pointer is no longer needed. @param fs The configured filesystem handle. @param path The path of the encryption zone. @return Returns a dynamically-allocated hdfsEncryptionZoneInfo object; NULL on error.

Get an array containing hostnames, offset and size of portions of the given file.

If hdfs is configured with HA namenode, return all namenode informations as an array. Else return NULL.

If hdfs is configured with HA namenode, return all namenode informations as an array. Else return NULL.

hdfsGetHosts - Get hostnames where a particular block (determined by pos & blocksize) of a file is stored. The last element in the array is NULL. Due to replication, a single block could be present on multiple hosts. @param fs The configured filesystem handle. @param path The path of the file. @param start The start of the block. @param length The length of the block. @return Returns a dynamically-allocated 2-d array of blocks-hosts; NULL on error.

Return error information of last failed operation.

hdfsGetPathInfo - Get information about a path as a (dynamically allocated) single hdfsFileInfo struct. hdfsFreeFileInfo should be called when the pointer is no longer needed. @param fs The configured filesystem handle. @param path The path of the file. @return Returns a dynamically-allocated hdfsFileInfo object; NULL on error.

hdfsGetUsed - Return the total raw size of all files in the filesystem. @param fs The configured filesystem handle. @return Returns the total-size; -1 on error.

hdfsGetWorkingDirectory - Get the current working directory for the given filesystem. @param fs The configured filesystem handle. @param buffer The user-buffer to copy path of cwd into. @param bufferSize The length of user-buffer. @return Returns buffer, NULL on error.

hdfsHFlush - Flush out the data in client’s user buffer. After the return of this call, new readers will see the data. @param fs configured filesystem handle @param file file handle @return 0 on success, -1 on error and sets errno

hdfsListDirectory - Get list of files/directories for a given directory-path. hdfsFreeFileInfo should be called to deallocate memory. @param fs The configured filesystem handle. @param path The path of the directory. @param numEntries Set to the number of files/directories in path. @return Returns a dynamically-allocated array of hdfsFileInfo objects; NULL on error.

hdfsEncryptionZoneInfo - Get list of all the encryption zones. hdfsFreeEncryptionZoneInfo should be called to deallocate memory. @param fs The configured filesystem handle. @return Returns a dynamically-allocated array of hdfsEncryptionZoneInfo objects; NULL on error.

hdfsMove - Move file from one filesystem to another. @param srcFS The handle to source filesystem. @param src The path of source file. @param dstFS The handle to destination filesystem. @param dst The path of destination file. @return Returns 0 on success, -1 on error.

Create an HDFS builder.

hdfsOpenFile - Open a hdfs file in given mode. @param fs The configured filesystem handle. @param path The full path to the file. @param flags - an | of bits/fcntl.h file flags - supported flags are O_RDONLY, O_WRONLY (meaning create or overwrite i.e., implies O_TRUNCAT), O_WRONLY|O_APPEND and O_SYNC. Other flags are generally ignored other than (O_RDWR || (O_EXCL & O_CREAT)) which return NULL and set errno equal ENOTSUP. @param bufferSize Size of buffer for read/write - pass 0 if you want to use the default configured values. @param replication Block replication - pass 0 if you want to use the default configured values. @param blocksize Size of block - pass 0 if you want to use the default configured values. @return Returns the handle to the open file or NULL on error.

hdfsRead - Read data from an open file. @param fs The configured filesystem handle. @param file The file handle. @param buffer The buffer to copy read bytes into. @param length The length of the buffer. @return On success, a positive number indicating how many bytes were read. On end-of-file, 0. On error, -1. Errno will be set to the error code. Just like the POSIX read function, hdfsRead will return -1 and set errno to EINTR if data is temporarily unavailable, but we are not yet at the end of the file.

hdfsRename - Rename file. @param fs The configured filesystem handle. @param oldPath The path of the source file. @param newPath The path of the destination file. @return Returns 0 on success, -1 on error.

Renew a delegation token.

hdfsSeek - Seek to given offset in file. This works only for files opened in read-only mode. @param fs The configured filesystem handle. @param file The file handle. @param desiredPos Offset into the file to seek into. @return Returns 0 on success, -1 on error.

hdfsSetReplication - Set the replication of the specified file to the supplied value @param fs The configured filesystem handle. @param path The path of the file. @return Returns 0 on success, -1 on error.

hdfsSetWorkingDirectory - Set the working directory. All relative paths will be resolved relative to it. @param fs The configured filesystem handle. @param path The path of the new ‘cwd’. @return Returns 0 on success, -1 on error.

hdfsSync - Flush out and sync the data in client’s user buffer. After the return of this call, new readers will see the data. @param fs configured filesystem handle @param file file handle @return 0 on success, -1 on error and sets errno

hdfsTell - Get the current offset in the file, in bytes. @param fs The configured filesystem handle. @param file The file handle. @return Current offset, -1 on error.

hdfsTruncate - Truncate the file in the indicated path to the indicated size. @param fs The configured filesystem handle. @param path the path to the file. @param pos the position the file will be truncated to. @param shouldWait output value, true if and client does not need to wait for block recovery, false if client needs to wait for block recovery.

hdfsUtime @param fs The configured filesystem handle. @param path the path to the file or directory @param mtime new modification time or -1 for no change @param atime new access time or -1 for no change @return 0 on success else -1

hdfsWrite - Write data into an open file. @param fs The configured filesystem handle. @param file The file handle. @param buffer The data. @param length The no. of bytes to write. @return Returns the number of bytes written, -1 on error.

Type Definitions

time type in seconds

offset within the file

Some utility decls used in libhdfs.

size of data for read/write io ops