pub struct SinusoidalEncoding<T: Scalar, B: ComputeBackend + Default = MoiraiBackend> {
pub table: Tensor<T, B>,
pub max_len: usize,
pub d_model: usize,
}Expand description
Sinusoidal (non-learnable) positional encoding layer.
Precomputes a [max_len, d_model] table and adds a row-slice of it to
the input at forward time.
Fields§
§table: Tensor<T, B>Precomputed PE table: [max_len, d_model].
max_len: usizeMaximum sequence length the precomputed table covers.
d_model: usizeEmbedding dimension (must be even).
Implementations§
Source§impl<T: Float + FloatElement, B: ComputeBackend + Default> SinusoidalEncoding<T, B>
impl<T: Float + FloatElement, B: ComputeBackend + Default> SinusoidalEncoding<T, B>
Trait Implementations§
Source§impl<T: Float + RealScalar, B: ElementwiseOps<T> + ReductionOps<T> + Default> Module<T, B> for SinusoidalEncoding<T, B>
impl<T: Float + RealScalar, B: ElementwiseOps<T> + ReductionOps<T> + Default> Module<T, B> for SinusoidalEncoding<T, B>
Source§fn forward(&self, input: &Var<T, B>) -> Result<Var<T, B>, ModuleError<B::Error>>
fn forward(&self, input: &Var<T, B>) -> Result<Var<T, B>, ModuleError<B::Error>>
Add positional encoding to input.
input:[batch, seq_len, d_model]
Returns [batch, seq_len, d_model].
Source§fn parameters(&self) -> Vec<Var<T, B>>
fn parameters(&self) -> Vec<Var<T, B>>
Collect all trainable parameters (including from sub-modules).
Source§fn named_parameters(&self) -> Vec<Parameter<T, B>>
fn named_parameters(&self) -> Vec<Parameter<T, B>>
Collect trainable parameters with stable hierarchical names. Read more
Source§fn load_parameters(&mut self, _params: &[Var<T, B>])
fn load_parameters(&mut self, _params: &[Var<T, B>])
Write optimizer-updated parameter values back into this module’s own
fields, consuming
params in the same order parameters() enumerates
them (i.e. params.len() == parameters().len()). Read moreSource§fn load_named_parameters(
&mut self,
parameters: &[Parameter<T, B>],
) -> Result<(), ParameterLoadError>
fn load_named_parameters( &mut self, parameters: &[Parameter<T, B>], ) -> Result<(), ParameterLoadError>
Load optimizer-updated named parameters after validating the complete
stable path inventory. Read more
Auto Trait Implementations§
impl<T, B> Forget for SinusoidalEncoding<T, B>
impl<T, B> Freeze for SinusoidalEncoding<T, B>
impl<T, B> Move for SinusoidalEncoding<T, B>
impl<T, B> RefUnwindSafe for SinusoidalEncoding<T, B>where
Tensor<T, B>: RefUnwindSafe,
impl<T, B> Send for SinusoidalEncoding<T, B>
impl<T, B> Sync for SinusoidalEncoding<T, B>
impl<T, B> Unpin for SinusoidalEncoding<T, B>
impl<T, B> UnsafeUnpin for SinusoidalEncoding<T, B>where
Tensor<T, B>: UnsafeUnpin,
impl<T, B> UnwindSafe for SinusoidalEncoding<T, B>where
Tensor<T, B>: UnwindSafe,
Blanket Implementations§
Source§impl<T> ArchivePointee for T
impl<T> ArchivePointee for T
Source§type ArchivedMetadata = ()
type ArchivedMetadata = ()
The archived version of the pointer metadata for this type.
Source§fn pointer_metadata(
_: &<T as ArchivePointee>::ArchivedMetadata,
) -> <T as Pointee>::Metadata
fn pointer_metadata( _: &<T as ArchivePointee>::ArchivedMetadata, ) -> <T as Pointee>::Metadata
Converts some archived metadata to the pointer metadata for itself.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> LayoutRaw for T
impl<T> LayoutRaw for T
Source§fn layout_raw(_: <T as Pointee>::Metadata) -> Result<Layout, LayoutError>
fn layout_raw(_: <T as Pointee>::Metadata) -> Result<Layout, LayoutError>
Returns the layout of the type.
Source§impl<T, N1, N2> Niching<NichedOption<T, N1>> for N2
impl<T, N1, N2> Niching<NichedOption<T, N1>> for N2
Source§unsafe fn is_niched(niched: *const NichedOption<T, N1>) -> bool
unsafe fn is_niched(niched: *const NichedOption<T, N1>) -> bool
Returns whether the given value has been niched. Read more
Source§fn resolve_niched(out: Place<NichedOption<T, N1>>)
fn resolve_niched(out: Place<NichedOption<T, N1>>)
Writes data to
out indicating that a T is niched.