Struct MapReduceSpec 

pub struct MapReduceSpec { /* private fields */ }

A map-reduce operation.

The mapper is optional: without one, the input is fed straight to the reducer, which is how YTsaurus models a plain shuffle-and-reduce.

Implementations

impl MapReduceSpec

pub fn new<I, O, K>( reducer: impl Into<String>, inputs: I, outputs: O, reduce_by: K, ) -> Self

where I: IntoIterator, I::Item: Into<String>, O: IntoIterator, O::Item: Into<String>, K: IntoIterator, K::Item: Into<String>,

A map-reduce running reducer over inputs, grouped by reduce_by.

pub fn with_mapper(self, command: impl Into<String>) -> Self

Adds a mapper phase.

pub fn with_local_file(self, path: impl Into<String>) -> Self

Adds a Cypress file to both phases.

One binary usually serves both, dispatching on argv[1], so attaching it to each phase separately would only be a way to forget one.

pub fn with_memory_limit(self, bytes: i64) -> Self

Sets the memory limit for both phases, in bytes.

pub fn with_sort_by<K>(self, columns: K) -> Self

where K: IntoIterator, K::Item: Into<String>,

Sets the sort columns, when they differ from the reduce columns.

pub fn without_key_switch(self) -> Self

Turns off key_switch delivery to the reducer.

Only useful for a reducer that does not group — with it off, JobReader::groups sees the whole input as one group.

pub fn with_raw(self, key: impl Into<String>, value: YsonValue) -> Self

Sets any spec field this builder does not model.

pub fn to_yson(&self) -> YsonValue

Renders the spec.

Trait Implementations

impl Clone for MapReduceSpec

fn clone(&self) -> MapReduceSpec

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MapReduceSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.