Module efflux::context

Hadoop job context representations and bindings.

This module exposes an arbitrarily typed map to be used as a job context for all Hadoop stages. It can be used to lookup different types and store state across executions of a task (although note that it's local to each mapper/reduce process). Authors of Mapper and Reducer implementations shouldn't need to store state here as they have mutable access to their struct values.

Values can be references as mut when required, as there should be only as single thread owning a Context at any given time. An example of inserting a value and retrieving it is as follows:

use efflux::prelude::*;

// custom state
#[derive(Eq, PartialEq)]
struct MyState {
    inner: usize
}

// only `Contextual` structs can be store
impl Contextual for MyState {}

// create a new context
let mut ctx = Context::new();

// create the state
let state = MyState { inner: 3 };

// store in context
ctx.insert(state);

// get a reference back out, as an option
let state_ref = ctx.get::<MyState>().expect("state not found");

// check it's the same state
assert_eq!(state_ref.inner, 3)

There are several types which will exist on a Context at various times throughout execution due to internal use. Whilst these can be read by the developer, they should rarely ever be modified as things may break. The current set of Contextual types added are as follows:

• Configuration
• Delimiters
• Offset

The most interesting of these types is the Configuration type, as it represents the job configuration provided by Hadoop.

Structs

Configuration	Configuration struct to represent a Hadoop configuration.
Context	Context structure to represent a Hadoop job context.
Delimiters	Delimiters struct to store the input/output separators for all stages of a MapReduce lifecycle. Once created, this structure should be considered immutable.
Offset	Offset structure to allow tracking of the current byte.

Traits

Contextual

Marker trait to represent types which can be added to a Context.