| Applies ‘{op}’ to each segment of the
| input tensor. Segments are defined
| by their LENGTHS. LENGTHS is a vector
| that maps each of the slices of
|
| DATA* to a particular segment. Values
| belonging to the same segment are aggregated
| together and considered for the ‘{op}’
| operation.
|
| For example LENGTHS = [2, 1] stands
| for segments DATA[0..1] and DATA[2]
|
| The sum of elements in LENGTHS must
| equal the number of elements in the first
| dimension of DATA. The length of OUTPUT
| is equal to the number of input segments,
| i.e. len(LENGTHS).
|
| {op_doc}
|
| {extra}
|
| bool GradientNeedIndices = false>
|
| Some notice:
|
| 1. Gradient actually doesn’t depend
| on whether sparse lookup is fused or
| not
|
| 2. INDICES are not used in CPU version,
| but they are needed in async CUDA version.
| So we register 3 input version for CPU
| as gradient op for
|
| GPU/CPU convert. We then register 2
| input version for CPU for backward compatibility
| with older nets.
|
| bool GradientNeedIndices = false>
|
| @brief Segment reduction op with optional fused
| embedding lookup
|
| Base implementation for LengthsXXX and
| SparseLengthsXXX depending on SparseFused static
| argument.
|
| Inputs:
| 0: DATA - input embedding to do lookups in
| 1..P: AUX_ARG_ - optional additional arguments to be passed to the
| reducer, should have the same first dimension as
| LENGTHS (e.g. scalars in WeightedSum)
| # if SparseFused == true:
| P+1: INDICES - 1-D vector with indices to look up in DATA. Should have the
| same dimension as LENGTHS
| # P+1 if SparseFused == false:
| P+1 or P+2: LENGTHS - lengths on indecies vector
|
| Output:
| Tensor with first dimension of K, where K = len(LENGTHS). Rest
| of dimensions are decided by reducer but usually are the same size as extra
| dimensions of DATA
|
| bool SparseFused = true,
| class InputAccessor = BaseInputAccessor>
|
| TODO(dzhulgakov): for now it’s implemented with
| incremental reducers because of fused sparse
| support. But using “lengths” representation
| actually implies continuous segments and thus
| range reducers can be used for non-sparse
| version.
| Version of gradient that requires the
| main input as well as the output of the
| forward op.
|
| Version of gradient that requires the main input
| and thus needs to receive length, indices and
| other stuff
|
| bool SparseFused = true,
| bool GradientNeedIndices = false>
| Reduces the input tensor along the last
| dimension of the input tensor by applying
| ‘{op}’. This op acts in a similar way
| to SortedSegment{op} and
|
| UnsortedSegment{op} but as if all input
| slices belong to a single segment.
|
| {op_doc}
|
| Reduces the input tensor along the first
| dimension of the input tensor by applying
| ‘{op}’. This op acts in a similar way
| to SortedSegment{op} and
|
| UnsortedSegment{op} but as if all input
| slices belong to a single segment.
|
| {op_doc}
|
| bool FirstDim = true>
|
| @brief
|
| Simple non-segmented reduction over
| the first few dimensions of the tensor
|
| Inputs:
|
| 0: DATA - input embedding to do lookups
| in
|
| 1..P: AUX_ARG_ - optional additional
| arguments to be passed to the reducer
|
| Args: num_reduce_dim (default 1) -
| the number of dims in front of the tensor
| to reduce
|
| Output:
|
| Tensor without the first num_dim
| dimensions of DATA class InputAccessor
| = BaseInputAccessor>
|
| Applies ‘{op}’ to each segment of input
| tensor. Segments need to be sorted and
| contiguous. See also UnsortedSegment{op}
| that doesn’t have this requirement.
|
| SEGMENT_IDS is a vector that maps each
| of the first dimension slices of the
|
| DATA to a particular group (segment).
| Values belonging to the same segment
| are aggregated together.
|
| The first dimension of the output is
| equal to the number of input segments,
| i.e. SEGMENT_IDS[-1]+1. Other dimensions
| are inherited from the input tensor.
|
| {op_doc}
|
| Gradient actually doesn’t depend on
| whether sparse lookup is fused or not
|
| @brief Segment reduction op with optional fused
| embedding lookup
|
| Base implementation for SortedSegmentXXX and
| SparseSortedSegmentXXX depending on SparseFused
| static argument.
|
| Inputs:
| 0: DATA - input embedding to do lookups in
| 1..P: AUX_ARG_ - optional additional arguments to be passed to the
| reducer, should have the same first dimension as
| SEGMENT_IDS (e.g. scalars in WeightedSum)
| # if SparseFused == true:
| P+1: INDICES - 1-D vector with indices to look up in DATA. Should have the
| same dimension as SEGMENT_IDS
| # P+1 if SparseFused == false:
| P+1 or P+2: SEGMENT_IDS - sorted segment ids 1-D vector
|
| Output:
|
| Tensor with first dimension of K, where K is
| the max segment id + 1. Rest of dimensions are
| decided by reducer but usually are the same
| size as extra dimensions of DATA
|
| bool SparseFused = true,
| class InputAccessor = BaseInputAccessor>
| Applies ‘{op}’ to each segment of input
| tensor. In order to allow for more efficient
| implementation of ‘{op}’, the input
| segments have to be contiguous and non-empty.
|
| SEGMENT_IDS is a vector that maps each
| of the first dimension slices of the
|
| DATA to a particular group (segment).
| Values belonging to the same segment
| are aggregated together.
|
| The first dimension of the output is
| equal to the number of input segments,
| i.e. SEGMENT_IDS[-1]+1. Other dimensions
| are inherited from the input tensor.
|
| {op_doc}
|
| Base implementation for segment reduction
| op that leverages continuity of the
| data
|
| Assumes that segments are sorted and
| there are no skip indices class InputAccessor
| = BaseInputAccessor>
|
| Pulls in slices of the input tensor,
| groups them into segments and applies
| ‘{op}’ to each segment. Segments are
| defined by their LENGTHS.
|
| This op is basically Gather and Lengths{op}
| fused together.
|
| INDICES should contain integers in
| range 0..N-1 where N is the first dimension
| of DATA. INDICES represent which slices
| of DATA need to be pulled in.
|
| LENGTHS is a vector that defines slice
| sizes by first dimension of DATA. Values
| belonging to the same segment are aggregated
| together. sum(LENGTHS) has to match
| INDICES size.
|
| The first dimension of the output is
| equal to the number of input segment,
| i.e. len(LENGTHS). Other dimensions
| are inherited from the input tensor.
|
| {op_doc} bool GradientNeedIndices
| = false>
|
| Pulls in slices of the input tensor,
| groups them into segments and applies
| ‘{op}’ to each segment. Segments need
| to be sorted and contiguous. See also
|
| SparseUnsortedSegment{op} that doesn’t
| have this requirement.
|
| This op is basically Gather and SortedSegment{op}
| fused together.
|
| INDICES should contain integers in
| range 0..N-1 where N is the first dimension
| of DATA. INDICES represent which slices
| of DATA need to be pulled in.
|
| SEGMENT_IDS is a vector that maps each
| referenced slice of the DATA to a particular
| group (segment). Values belonging
| to the same segment are aggregated together.
| SEGMENT_IDS should have the same dimension
| as INDICES.
|
| The first dimension of the output is
| equal to the number of input segments,
| i.e. SEGMENT_IDS[-1]+1. Other dimensions
| are inherited from the input tensor.
|
| {op_doc}
|
| Pulls in slices of the input tensor,
| groups them into segments and applies
| ‘{op}’ to each segment. Segments ids
| can appear in arbitrary order (unlike
| in
|
| SparseSortedSegment{op}).
|
| This op is basically Gather and UnsortedSegment{op}
| fused together.
|
| INDICES should contain integers in
| range 0..N-1 where N is the first dimension
| of DATA. INDICES represent which slices
| of DATA need to be pulled in.
|
| SEGMENT_IDS is a vector that maps each
| referenced slice of the DATA to a particular
| group (segment). Values belonging
| to the same segment are aggregated together.
| SEGMENT_IDS should have the same dimension
| as INDICES.
|
| If num_segments argument is passed
| it would be used as a first dimension
| for the output. Otherwise, it’d be dynamically
| calculated from as the max value of
|
| SEGMENT_IDS plus one. Other output
| dimensions are inherited from the input
| tensor.
|
| {op_doc}
|
| Applies ‘{op}’ to each segment of input
| tensor. Segments ids can appear in arbitrary
| order (unlike in SortedSegment{op}).
|
| SEGMENT_IDS is a vector that maps each
| of the first dimension slices of the
|
| DATA to a particular group (segment).
| Values belonging to the same segment
| are aggregated together.
|
| If num_segments argument is passed
| it would be used as a first dimension
| for the output. Otherwise, it’d be dynamically
| calculated from as the max value of
|
| SEGMENT_IDS plus one. Other output
| dimensions are inherited from the input
| tensor.
|
| {op_doc}
|
| Gradient actually doesn’t depend on
| whether sparse lookup is fused or not
|
| @brief Unsorted segment reduction op with
| optional fused embedding lookup
|
| Base implementation for UnsortedSegmentXXX and
| UnsparseSortedSegmentXXX depending on
| SparseFused static argument.
|
| Unlike the sorted version it allows to have
| “gaps” in segment ids.
|
| Inputs:
| 0: DATA - input embedding to do lookups in
| 1..P: AUX_ARG_ - optional additional arguments to be passed to the
| reducer, should have the same first dimension as
| SEGMENT_IDS (e.g. scalars in WeightedSum)
| # if SparseFused == true:
| P+1: INDICES - 1-D vector with indices to look up in DATA. Should have the
| same dimension as SEGMENT_IDS
| # P+1 if SparseFused == false:
| P+1 or P+2: SEGMENT_IDS - unsorted segment ids 1-D vector
|
| Args:
| num_segments - allows to override the
| dimension of the output. If not set it would
| be inferred from segment_ids tensor.
|
|
| Output:
| Tensor with first dimension of K, where K is
| the max segment id + 1. Rest of dimensions are
| decided by reducer but usually are the same
| size as extra dimensions of DATA
|
| bool SparseFused = true,
| class InputAccessor = BaseInputAccessor>
| base implementation of sparse/non-sparse
| gradient computation
|
| bool GradientNeedIndices = false>
|
| base implementation of sorted/unsorted
| sparse/non-sparse gradient computation
|