Expand description
Efflux is a set of Rust interfaces for MapReduce and Hadoop Streaming.
This crate provides easy interfaces for working with MapReduce, whether
or not you’re running on the Hadoop platform. Usage is as simple as a
struct which implements either the Mapper
or Reducer
trait, as all
other interaction is taken care of internally.
Macros are provided for IO, to provide a compile-time guarantee of things such as counter/status updates, or writing to the Hadoop task logs.
Modules
Hadoop job context representations and bindings.
IO binding module for the
efflux
crate.Compile time utilities to ease Hadoop usage.
Exposed structures based on the mapping stage.
A “prelude” for crates using the
efflux
crate.Exposed structures based on the reduction stage.
Macros
Prints output to the Hadoop task logs.
Updates a counter for the current job.
Updates the status for the current job.
Functions
Executes a
Mapper
against the current stdin
.Executes a
Reducer
against the current stdin
.