# Crate caffe2op_segmentreduction

source ·## Macros

- | Helper macro when the main op is defined | elsewhere, and we only need to define the | schema, and the gradient op. | | TODO: enable input fillers

## Structs

- | Applies ‘{op}’ to each segment of the | input tensor. Segments are defined | by their
*LENGTHS*.*LENGTHS*is a vector | that maps each of the slices of | | DATA* to a particular segment. Values | belonging to the same segment are aggregated | together and considered for the ‘{op}’ | operation. | | For example*LENGTHS = [2, 1]*stands | for segments*DATA[0..1]*and*DATA[2]*| | The sum of elements in*LENGTHS*must | equal the number of elements in the first | dimension of*DATA*. The length of*OUTPUT*| is equal to the number of input segments, | i.e. len(*LENGTHS*). | | {op_doc} | | {extra} | | bool GradientNeedIndices = false> | - | Some notice: | | 1. Gradient actually doesn’t depend | on whether sparse lookup is fused or | not | | 2. INDICES are not used in CPU version, | but they are needed in async CUDA version. | So we register 3 input version for CPU | as gradient op for | | GPU/CPU convert. We then register 2 | input version for CPU for backward compatibility | with older nets. | | bool GradientNeedIndices = false> |
- | @brief Segment reduction op with optional fused | embedding lookup | | Base implementation for LengthsXXX and | SparseLengthsXXX depending on SparseFused static | argument. | | Inputs: | 0: DATA - input embedding to do lookups in | 1..P: AUX_ARG_
*- optional additional arguments to be passed to the | reducer, should have the same first dimension as | LENGTHS (e.g. scalars in WeightedSum) | # if SparseFused == true: | P+1: INDICES - 1-D vector with indices to look up in DATA. Should have the | same dimension as LENGTHS | # P+1 if SparseFused == false: | P+1 or P+2: LENGTHS - lengths on indecies vector | | Output: | Tensor with first dimension of K, where K = len(LENGTHS). Rest | of dimensions are decided by reducer but usually are the same size as extra | dimensions of DATA | | bool SparseFused = true, | class InputAccessor = BaseInputAccessor*> | | TODO(dzhulgakov): for now it’s implemented with | incremental reducers because of fused sparse | support. But using “lengths” representation | actually implies continuous segments and thus | range reducers can be used for non-sparse | version. *| Version of gradient that requires the | main input as well as the output of the | forward op. |**| Version of gradient that requires the main input | and thus needs to receive length, indices and | other stuff | | bool SparseFused = true, | bool GradientNeedIndices = false>**| Reduces the input tensor along the last | dimension of the input tensor by applying | ‘{op}’. This op acts in a similar way | to SortedSegment{op} and | | UnsortedSegment{op} but as if all input | slices belong to a single segment. | | {op_doc} |**| Reduces the input tensor along the first | dimension of the input tensor by applying | ‘{op}’. This op acts in a similar way | to SortedSegment{op} and | | UnsortedSegment{op} but as if all input | slices belong to a single segment. | | {op_doc} |**| bool FirstDim = true> |**| @brief | | Simple non-segmented reduction over | the first few dimensions of the tensor | | Inputs: | | 0: DATA - input embedding to do lookups | in | | 1..P: AUX_ARG_**- optional additional | arguments to be passed to the reducer | | Args: num_reduce_dim (default 1) - | the number of dims in front of the tensor | to reduce | | Output: | | Tensor without the first*`num_dim`

| dimensions of DATA class InputAccessor | = BaseInputAccessor> | *| Applies ‘{op}’ to each segment of input | tensor. Segments need to be sorted and | contiguous. See also UnsortedSegment{op} | that doesn’t have this requirement. | | SEGMENT_IDS is a vector that maps each | of the first dimension slices of the | | DATA to a particular group (segment). | Values belonging to the same segment | are aggregated together. | | The first dimension of the output is | equal to the number of input segments, | i.e.*`SEGMENT_IDS[-1]+1`

. Other dimensions | are inherited from the input tensor. | | {op_doc} |*| Gradient actually doesn’t depend on | whether sparse lookup is fused or not |**| @brief Segment reduction op with optional fused | embedding lookup | | Base implementation for SortedSegmentXXX and | SparseSortedSegmentXXX depending on SparseFused | static argument. | | Inputs: | 0: DATA - input embedding to do lookups in | 1..P: AUX_ARG_**- optional additional arguments to be passed to the | reducer, should have the same first dimension as | SEGMENT_IDS (e.g. scalars in WeightedSum) | # if SparseFused == true: | P+1: INDICES - 1-D vector with indices to look up in DATA. Should have the | same dimension as SEGMENT_IDS | # P+1 if SparseFused == false: | P+1 or P+2: SEGMENT_IDS - sorted segment ids 1-D vector | | Output: | | Tensor with first dimension of K, where K is | the max segment id + 1. Rest of dimensions are | decided by reducer but usually are the same | size as extra dimensions of DATA | | bool SparseFused = true, | class InputAccessor = BaseInputAccessor*> *| Applies ‘{op}’ to each segment of input | tensor. In order to allow for more efficient | implementation of ‘{op}’, the input | segments have to be contiguous and non-empty. | | SEGMENT_IDS is a vector that maps each | of the first dimension slices of the | | DATA to a particular group (segment). | Values belonging to the same segment | are aggregated together. | | The first dimension of the output is | equal to the number of input segments, | i.e.*`SEGMENT_IDS[-1]+1`

. Other dimensions | are inherited from the input tensor. | | {op_doc} |*| Base implementation for segment reduction | op that leverages continuity of the | data | | Assumes that segments are sorted and | there are no skip indices class InputAccessor | = BaseInputAccessor*> | *| Pulls in slices of the input tensor, | groups them into segments and applies | ‘{op}’ to each segment. Segments are | defined by their LENGTHS. | | This op is basically Gather and Lengths{op} | fused together. | | INDICES should contain integers in | range 0..N-1 where N is the first dimension | of DATA. INDICES represent which slices | of DATA need to be pulled in. | | LENGTHS is a vector that defines slice | sizes by first dimension of DATA. Values | belonging to the same segment are aggregated | together. sum(LENGTHS) has to match | INDICES size. | | The first dimension of the output is | equal to the number of input segment, | i.e.*`len(LENGTHS)`

. Other dimensions | are inherited from the input tensor. | | {op_doc} bool GradientNeedIndices | = false> |*| Pulls in slices of the input tensor, | groups them into segments and applies | ‘{op}’ to each segment. Segments need | to be sorted and contiguous. See also | | SparseUnsortedSegment{op} that doesn’t | have this requirement. | | This op is basically Gather and SortedSegment{op} | fused together. | | INDICES should contain integers in | range 0..N-1 where N is the first dimension | of DATA. INDICES represent which slices | of DATA need to be pulled in. | | SEGMENT_IDS is a vector that maps each | referenced slice of the DATA to a particular | group (segment). Values belonging | to the same segment are aggregated together. | SEGMENT_IDS should have the same dimension | as INDICES. | | The first dimension of the output is | equal to the number of input segments, | i.e.*`SEGMENT_IDS[-1]+1`

. Other dimensions | are inherited from the input tensor. | | {op_doc} |*| Pulls in slices of the input tensor, | groups them into segments and applies | ‘{op}’ to each segment. Segments ids | can appear in arbitrary order (unlike | in | | SparseSortedSegment{op}). | | This op is basically Gather and UnsortedSegment{op} | fused together. | | INDICES should contain integers in | range 0..N-1 where N is the first dimension | of DATA. INDICES represent which slices | of DATA need to be pulled in. | | SEGMENT_IDS is a vector that maps each | referenced slice of the DATA to a particular | group (segment). Values belonging | to the same segment are aggregated together. | SEGMENT_IDS should have the same dimension | as INDICES. | | If*`num_segments`

argument is passed | it would be used as a first dimension | for the output. Otherwise, it’d be dynamically | calculated from as the max value of | | SEGMENT_IDS plus one. Other output | dimensions are inherited from the input | tensor. | | {op_doc} |*| Applies ‘{op}’ to each segment of input | tensor. Segments ids can appear in arbitrary | order (unlike in SortedSegment{op}). | | SEGMENT_IDS is a vector that maps each | of the first dimension slices of the | | DATA to a particular group (segment). | Values belonging to the same segment | are aggregated together. | | If*`num_segments`

argument is passed | it would be used as a first dimension | for the output. Otherwise, it’d be dynamically | calculated from as the max value of | | SEGMENT_IDS plus one. Other output | dimensions are inherited from the input | tensor. | | {op_doc} |*| Gradient actually doesn’t depend on | whether sparse lookup is fused or not |**| @brief Unsorted segment reduction op with | optional fused embedding lookup | | Base implementation for UnsortedSegmentXXX and | UnsparseSortedSegmentXXX depending on | SparseFused static argument. | | Unlike the sorted version it allows to have | “gaps” in segment ids. | | Inputs: | 0: DATA - input embedding to do lookups in | 1..P: AUX_ARG_**- optional additional arguments to be passed to the | reducer, should have the same first dimension as | SEGMENT_IDS (e.g. scalars in WeightedSum) | # if SparseFused == true: | P+1: INDICES - 1-D vector with indices to look up in DATA. Should have the | same dimension as SEGMENT_IDS | # P+1 if SparseFused == false: | P+1 or P+2: SEGMENT_IDS - unsorted segment ids 1-D vector | | Args: | num_segments - allows to override the | dimension of the output. If not set it would | be inferred from segment_ids tensor. | | | Output: | Tensor with first dimension of K, where K is | the max segment id + 1. Rest of dimensions are | decided by reducer but usually are the same | size as extra dimensions of DATA | | bool SparseFused = true, | class InputAccessor = BaseInputAccessor*> *| base implementation of sparse/non-sparse | gradient computation | | bool GradientNeedIndices = false> |**| base implementation of sorted/unsorted | sparse/non-sparse gradient computation |*

*Enums*

*Enums**| figure out what the two comments below |**break*, if anything |*| TODO: figure out what the two comments | below break*, if anything |**| Input layout: | | orig_arg1, orig_arg2, …, orig_argN, | SEGMENT_GRADS, SEGMENT_IDS | | orig_argXs represent original op’s inputs and will | be passed to the reducer directly | | TODO: does the below comment break something?**| TODO: what do the below two comments |**break*, if anything? |

*Traits*

*Traits**| Incremental reducer ops: assume that | reducer consumes pieces of data one | by one. Also, supports additional arguments | passed to reducer, e.g. scalers for | weighted sum. | | ———– | @note | | in current implementation additional | inputs are considered auxiliary constants | and have limitations: | | - there is no gradient computation for | auxiliary inputs | | - auxiliary inputs aren’t affected | by fused embedding lookup in operations | like sparse_sorted_segment |*

*Functions*

*Functions**| Helper function to enforce naming conventions | at compile time. |*

*Type Definitions*

*Type Definitions*