1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
//! # Dropout-family operations
//!
//! Regularizers that zero part of an activation at random, as functions.
//! Each one takes its settings as arguments and runs whenever it is called;
//! deciding to run only while training is the caller's job. The matching
//! modules in [`crate::blocks::images::drop`] (and burn's [`Dropout`]) make
//! that decision: they apply the op only when autodiff is enabled.
//!
//! - [`dropout`] drops independent elements at random and rescales the rest by
//! `1 / (1 - prob)`; at `prob == 1` it returns zeros. It is the operation
//! inside burn's [`Dropout`] module, without the training check (burn's
//! divides by zero at `prob == 1`);
//! [`scaled_dot_product_attention`](crate::ops::transformers::attention::scaled_dot_product_attention)
//! uses it on attention weights.
//! - [`drop_path`] is stochastic depth: it drops whole batch rows, so a
//! residual branch is skipped for that sample. The [`DropPath`] module wraps
//! it.
//! - [`drop_block_2d`] is `DropBlock`: it drops contiguous spatial blocks of a
//! `[batch, channels, height, width]` map. Its settings are one
//! [`DropBlockOptions`] value, whose block size is a [`SizeConfig`] per axis
//! and whose dropped regions can be refilled with
//! [`NoiseConfig`](crate::ops::noise::NoiseConfig) noise. The [`DropBlock2d`]
//! module holds a `DropBlockOptions` and calls it.
//!
//! Why the structured drops regularize, and the rate tables that configure
//! a stack of `DropPath` layers, are in [`crate::blocks::images::drop`].
//!
//! [`Dropout`]: burn::nn::Dropout
//! [`DropPath`]: crate::blocks::images::drop::drop_path::DropPath
//! [`DropBlock2d`]: crate::blocks::images::drop::drop_block::DropBlock2d
pub use *;
pub use *;
pub use *;
pub use *;