Crate efflux

Efflux is a set of Rust interfaces for MapReduce and Hadoop Streaming.

This crate provides easy interfaces for working with MapReduce, whether or not you’re running on the Hadoop platform. Usage is as simple as a struct which implements either the Mapper or Reducer trait, as all other interaction is taken care of internally.

Macros are provided for IO, to provide a compile-time guarantee of things such as counter/status updates, or writing to the Hadoop task logs.

Modules

context

Hadoop job context representations and bindings.

io

IO binding module for the efflux crate.

macros

Compile time utilities to ease Hadoop usage.

mapper

Exposed structures based on the mapping stage.

prelude

A “prelude” for crates using the efflux crate.

reducer

Exposed structures based on the reduction stage.

Macros

log

Prints output to the Hadoop task logs.

update_counter

Updates a counter for the current job.

update_status

Updates the status for the current job.

Functions

run_mapper

Executes a Mapper against the current stdin.

run_reducer

Executes a Reducer against the current stdin.