Crate libhdfs3_sys

Source

Modules§

err
hdfs3

Structs§

BlockLocation
HdfsFileInternalWrapper
HdfsFileSystemInternalWrapper
Namenode
hdfsBuilder
hdfsEncryptionFileInfo
hdfsEncryptionFileInfo - Information about an encryption file/directory.
hdfsEncryptionZoneInfo
hdfsEncryptionZoneInfo- Information about an encryption zone.
hdfsFileInfo
hdfsFileInfo - Information about a file/directory.

Enums§

tObjectKind
port

Functions§

hdfsAvailable
hdfsAvailable - Number of bytes that can be read from this input stream without blocking. @param fs The configured filesystem handle. @param file The file handle. @return Returns available bytes; -1 on error.
hdfsBuilderConfSetStr
Set a configuration string for an HdfsBuilder.
hdfsBuilderConnect
Connect to HDFS using the parameters defined by the builder.
hdfsBuilderSetForceNewInstance
Do nothing, we always create a new instance
hdfsBuilderSetKerbTicketCachePath
Set the path to the Kerberos ticket cache to use when connecting to the HDFS cluster.
hdfsBuilderSetNameNode
Set the HDFS NameNode to connect to.
hdfsBuilderSetNameNodePort
Set the port of the HDFS NameNode to connect to.
hdfsBuilderSetToken
Set the token used to authenticate
hdfsBuilderSetUserName
Set the username to use when connecting to the HDFS cluster.
hdfsCancelDelegationToken
Cancel a delegation token.
hdfsChmod
hdfsChmod @param fs The configured filesystem handle. @param path the path to the file or directory @param mode the bitmask to set it to @return 0 on success else -1
hdfsChown
Change the user and/or group of a file or directory.
hdfsCloseFile
hdfsCloseFile - Close an open file. @param fs The configured filesystem handle. @param file The file handle. @return Returns 0 on success, -1 on error. On error, errno will be set appropriately. If the hdfs file was valid, the memory associated with it will be freed at the end of this call, even if there was an I/O error.
hdfsConfGetInt
Get a configuration integer.
hdfsConfGetStr
Get a configuration string.
hdfsConfStrFree
Free a configuration string found with hdfsConfGetStr.
hdfsConnect
hdfsConnect - Connect to a hdfs file system. Connect to the hdfs. @param nn The NameNode. See hdfsBuilderSetNameNode for details. @param port The port on which the server is listening. @return Returns a handle to the filesystem or NULL on error. @deprecated Use hdfsBuilderConnect instead.
hdfsConnectAsUser
hdfsConnectAsUser - Connect to a hdfs file system as a specific user Connect to the hdfs. @param nn The NameNode. See hdfsBuilderSetNameNode for details. @param port The port on which the server is listening. @param user the user name (this is hadoop domain user). Or NULL is equivelant to hhdfsConnect(host, port) @return Returns a handle to the filesystem or NULL on error. @deprecated Use hdfsBuilderConnect instead.
hdfsConnectAsUserNewInstance
hdfsConnect - Connect to an hdfs file system.
hdfsConnectNewInstance
hdfsConnect - Connect to an hdfs file system.
hdfsCopy
hdfsCopy - Copy file from one filesystem to another. @param srcFS The handle to source filesystem. @param src The path of source file. @param dstFS The handle to destination filesystem. @param dst The path of destination file. @return Returns 0 on success, -1 on error.
hdfsCreateDirectory
hdfsCreateDirectory - Make the given file and all non-existent parents into directories. @param fs The configured filesystem handle. @param path The path of the directory. @return Returns 0 on success, -1 on error.
hdfsCreateEncryptionZone
Create encryption zone for the directory with specific key name @param fs The configured filesystem handle. @param path The path of the directory. @param keyname The key name of the encryption zone @return Returns 0 on success, -1 on error.
hdfsDelete
hdfsDelete - Delete file. @param fs The configured filesystem handle. @param path The path of the file. @param recursive if path is a directory and set to non-zero, the directory is deleted else throws an exception. In case of a file the recursive argument is irrelevant. @return Returns 0 on success, -1 on error.
hdfsDisconnect
hdfsDisconnect - Disconnect from the hdfs file system. Disconnect from hdfs. @param fs The configured filesystem handle. @return Returns 0 on success, -1 on error. Even if there is an error, the resources associated with the hdfsFS will be freed.
hdfsExists
hdfsExists - Checks if a given path exsits on the filesystem @param fs The configured filesystem handle. @param path The path to look for @return Returns 0 on success, -1 on error.
hdfsFileIsOpenForRead
Determine if a file is open for read.
hdfsFileIsOpenForWrite
Determine if a file is open for write.
hdfsFlush
hdfsWrite - Flush the data. @param fs The configured filesystem handle. @param file The file handle. @return Returns 0 on success, -1 on error.
hdfsFreeBuilder
Free an HDFS builder.
hdfsFreeDelegationToken
Free a delegation token.
hdfsFreeEncryptionZoneInfo
hdfsFreeEncryptionZoneInfo - Free up the hdfsEncryptionZoneInfo array (including fields) @param infos The array of dynamically-allocated hdfsEncryptionZoneInfo objects. @param numEntries The size of the array.
hdfsFreeFileBlockLocations
Free the BlockLocation array returned by hdfsGetFileBlockLocations
hdfsFreeFileInfo
hdfsFreeFileInfo - Free up the hdfsFileInfo array (including fields) @param infos The array of dynamically-allocated hdfsFileInfo objects. @param numEntries The size of the array.
hdfsFreeHosts
hdfsFreeHosts - Free up the structure returned by hdfsGetHosts @param hdfsFileInfo The array of dynamically-allocated hdfsFileInfo objects. @param numEntries The size of the array.
hdfsFreeNamenodeInformation
Free the array returned by hdfsGetConfiguredNamenodes()
hdfsGetCapacity
hdfsGetCapacity - Return the raw capacity of the filesystem. @param fs The configured filesystem handle. @return Returns the raw-capacity; -1 on error.
hdfsGetDefaultBlockSize
hdfsGetDefaultBlockSize - Get the default blocksize.
hdfsGetDelegationToken
Get a delegation token from namenode. The token should be freed using hdfsFreeDelegationToken after canceling the token or token expired.
hdfsGetEZForPath
hdfsEncryptionZoneInfo - Get information about a path as a (dynamically allocated) single hdfsEncryptionZoneInfo struct. hdfsEncryptionZoneInfo should be called when the pointer is no longer needed. @param fs The configured filesystem handle. @param path The path of the encryption zone. @return Returns a dynamically-allocated hdfsEncryptionZoneInfo object; NULL on error.
hdfsGetFileBlockLocations
Get an array containing hostnames, offset and size of portions of the given file.
hdfsGetHANamenodes
If hdfs is configured with HA namenode, return all namenode informations as an array. Else return NULL.
hdfsGetHANamenodesWithConfig
If hdfs is configured with HA namenode, return all namenode informations as an array. Else return NULL.
hdfsGetHosts
hdfsGetHosts - Get hostnames where a particular block (determined by pos & blocksize) of a file is stored. The last element in the array is NULL. Due to replication, a single block could be present on multiple hosts. @param fs The configured filesystem handle. @param path The path of the file. @param start The start of the block. @param length The length of the block. @return Returns a dynamically-allocated 2-d array of blocks-hosts; NULL on error.
hdfsGetLastError
Return error information of last failed operation.
hdfsGetPathInfo
hdfsGetPathInfo - Get information about a path as a (dynamically allocated) single hdfsFileInfo struct. hdfsFreeFileInfo should be called when the pointer is no longer needed. @param fs The configured filesystem handle. @param path The path of the file. @return Returns a dynamically-allocated hdfsFileInfo object; NULL on error.
hdfsGetUsed
hdfsGetUsed - Return the total raw size of all files in the filesystem. @param fs The configured filesystem handle. @return Returns the total-size; -1 on error.
hdfsGetWorkingDirectory
hdfsGetWorkingDirectory - Get the current working directory for the given filesystem. @param fs The configured filesystem handle. @param buffer The user-buffer to copy path of cwd into. @param bufferSize The length of user-buffer. @return Returns buffer, NULL on error.
hdfsHFlush
hdfsHFlush - Flush out the data in client’s user buffer. After the return of this call, new readers will see the data. @param fs configured filesystem handle @param file file handle @return 0 on success, -1 on error and sets errno
hdfsListDirectory
hdfsListDirectory - Get list of files/directories for a given directory-path. hdfsFreeFileInfo should be called to deallocate memory. @param fs The configured filesystem handle. @param path The path of the directory. @param numEntries Set to the number of files/directories in path. @return Returns a dynamically-allocated array of hdfsFileInfo objects; NULL on error.
hdfsListEncryptionZones
hdfsEncryptionZoneInfo - Get list of all the encryption zones. hdfsFreeEncryptionZoneInfo should be called to deallocate memory. @param fs The configured filesystem handle. @return Returns a dynamically-allocated array of hdfsEncryptionZoneInfo objects; NULL on error.
hdfsMove
hdfsMove - Move file from one filesystem to another. @param srcFS The handle to source filesystem. @param src The path of source file. @param dstFS The handle to destination filesystem. @param dst The path of destination file. @return Returns 0 on success, -1 on error.
hdfsNewBuilder
Create an HDFS builder.
hdfsOpenFile
hdfsOpenFile - Open a hdfs file in given mode. @param fs The configured filesystem handle. @param path The full path to the file. @param flags - an | of bits/fcntl.h file flags - supported flags are O_RDONLY, O_WRONLY (meaning create or overwrite i.e., implies O_TRUNCAT), O_WRONLY|O_APPEND and O_SYNC. Other flags are generally ignored other than (O_RDWR || (O_EXCL & O_CREAT)) which return NULL and set errno equal ENOTSUP. @param bufferSize Size of buffer for read/write - pass 0 if you want to use the default configured values. @param replication Block replication - pass 0 if you want to use the default configured values. @param blocksize Size of block - pass 0 if you want to use the default configured values. @return Returns the handle to the open file or NULL on error.
hdfsRead
hdfsRead - Read data from an open file. @param fs The configured filesystem handle. @param file The file handle. @param buffer The buffer to copy read bytes into. @param length The length of the buffer. @return On success, a positive number indicating how many bytes were read. On end-of-file, 0. On error, -1. Errno will be set to the error code. Just like the POSIX read function, hdfsRead will return -1 and set errno to EINTR if data is temporarily unavailable, but we are not yet at the end of the file.
hdfsRename
hdfsRename - Rename file. @param fs The configured filesystem handle. @param oldPath The path of the source file. @param newPath The path of the destination file. @return Returns 0 on success, -1 on error.
hdfsRenewDelegationToken
Renew a delegation token.
hdfsSeek
hdfsSeek - Seek to given offset in file. This works only for files opened in read-only mode. @param fs The configured filesystem handle. @param file The file handle. @param desiredPos Offset into the file to seek into. @return Returns 0 on success, -1 on error.
hdfsSetReplication
hdfsSetReplication - Set the replication of the specified file to the supplied value @param fs The configured filesystem handle. @param path The path of the file. @return Returns 0 on success, -1 on error.
hdfsSetWorkingDirectory
hdfsSetWorkingDirectory - Set the working directory. All relative paths will be resolved relative to it. @param fs The configured filesystem handle. @param path The path of the new ‘cwd’. @return Returns 0 on success, -1 on error.
hdfsSync
hdfsSync - Flush out and sync the data in client’s user buffer. After the return of this call, new readers will see the data. @param fs configured filesystem handle @param file file handle @return 0 on success, -1 on error and sets errno
hdfsTell
hdfsTell - Get the current offset in the file, in bytes. @param fs The configured filesystem handle. @param file The file handle. @return Current offset, -1 on error.
hdfsTruncate
hdfsTruncate - Truncate the file in the indicated path to the indicated size. @param fs The configured filesystem handle. @param path the path to the file. @param pos the position the file will be truncated to. @param shouldWait output value, true if and client does not need to wait for block recovery, false if client needs to wait for block recovery.
hdfsUtime
hdfsUtime @param fs The configured filesystem handle. @param path the path to the file or directory @param mtime new modification time or -1 for no change @param atime new access time or -1 for no change @return 0 on success else -1
hdfsWrite
hdfsWrite - Write data into an open file. @param fs The configured filesystem handle. @param file The file handle. @param buffer The data. @param length The no. of bytes to write. @return Returns the number of bytes written, -1 on error.

Type Aliases§

__int16_t
__int32_t
__int64_t
__time_t
__uint16_t
__uint32_t
hdfsFS
hdfsFile
size_t
tOffset
time type in seconds
tPort
offset within the file
tSize
Some utility decls used in libhdfs.
tTime
size of data for read/write io ops
time_t