Skip to main content

Dropout

Struct Dropout 

Source
pub struct Dropout {
    pub p: f32,
    pub training: bool,
}
Expand description

Randomly zeroes elements with probability p during training and scales the rest by 1/(1 − p); the identity in evaluation mode (MLX’s nn.Dropout). The mask is bernoulli(1 − p), drawn from the thread’s key sequence (bit for bit MLX’s for a seed) or from an explicit key (Dropout::forward_with_key).

The mode is switched with Module::train on the whole model, and is part of the model’s structure, so a compiled step retraces when it changes.

Fields§

§p: f32

The probability of zeroing an element, 0 ≤ p < 1.

§training: bool

Training (true, the default) or evaluation mode.

Implementations§

Source§

impl Dropout

Source

pub fn new(p: f32) -> Self

Dropout with probability p (0 ≤ p < 1; another value makes forward return a failed array), in training mode.

Source

pub fn forward(&self, x: &Array) -> Array

x with dropout (training mode) or unchanged (evaluation mode).

Source

pub fn forward_with_key(&self, x: &Array, key: &Array) -> Array

Dropout::forward with the mask drawn from key.

Trait Implementations§

Source§

impl Clone for Dropout

Source§

fn clone(&self) -> Self

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for Dropout

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Leaves for Dropout

Source§

fn visit(&self, __v: &mut Visit<'_>)

Reports every leaf (and child tree, enum variant, mode flag and static field) to v, in order.
Source§

impl Tree for Dropout
where Self: Clone,

Source§

fn visit_mut(&mut self, __v: &mut VisitMut<'_>)

Reports every leaf (and child tree and mode flag) to v mutably, in the order of Leaves::visit.
Source§

fn for_each(&self, f: impl FnMut(&str, &Array))

Calls f(path, leaf) for every leaf, in order.
Source§

fn map(&self, f: impl FnMut(&str, &Array) -> Array) -> Self

The tree with every leaf replaced by f(path, leaf) (same structure; parameters keep their frozen flag).
Source§

fn leaves(&self) -> Vec<Array>

The leaves, in order.
Source§

fn paths(&self) -> Vec<String>

The leaves’ paths, in order.
Source§

fn flatten(&self) -> Vec<(String, Array)>

The leaves with their paths, in order (MLX’s tree_flatten).
Source§

fn unflatten( &self, leaves: impl IntoIterator<Item = Array>, ) -> Result<Self, Error>

A tree of this structure (self is the template) with leaves in order: exactly Tree::len of them, else an error naming both counts.
Source§

fn load_flat( &mut self, named: impl IntoIterator<Item = (String, Array)>, mode: LoadMode, ) -> Result<(), Error>

Replaces leaves by path. LoadMode::Strict requires exactly the tree’s paths with matching shapes; LoadMode::Partial assigns the matching names and ignores unknown ones (shapes must still match). Loaded dtypes are kept. On any problem the error lists every one (missing, unknown and mismatched names, sorted) and the tree is unchanged. Duplicate paths (in the tree or in named) are an error.
Source§

fn len(&self) -> usize

The number of leaves.
Source§

fn is_empty(&self) -> bool

Whether the tree has no leaves.
Source§

fn def(&self) -> TreeDef

The tree’s structure.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> AutoreleaseSafe for T
where T: ?Sized,

Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> IntoResult for T
where T: Tree,

Source§

type Ok = T

The value.
Source§

fn into_result(self) -> Result<T, Error>

The value, or the closure’s error.
Source§

impl<T> Module for T
where T: Tree,

Source§

fn parameters(&self) -> Vec<(String, Array)>

Every leaf (parameters, buffers and plain arrays) with its path, in order.
Source§

fn trainable_parameters(&self) -> Vec<(String, Array)>

The trainable leaves with their paths, in order.
Source§

fn num_params(&self) -> usize

The number of elements over every leaf (MLX’s tree_flatten total).
Source§

fn num_trainable(&self) -> usize

The number of elements over the trainable leaves.
Source§

fn summary(&self) -> Summary

A table of the leaves (path, shape, dtype, trainable, bytes) with totals.
Source§

fn display(&self) -> ModuleDisplay<'_, Self>

The nested modules with their parameter counts, for printing: println!("{}", model.display()).
Source§

fn freeze(&mut self, filter: impl Fn(&str) -> bool) -> Result<usize>

Freezes the parameters whose path matches filter, returning how many changed. A matching plain Array leaf cannot be frozen (only Params can): that is an error naming the paths, and nothing changes. Buffers are never trainable anyway.
Source§

fn unfreeze(&mut self, filter: impl Fn(&str) -> bool) -> Result<usize>

Unfreezes the parameters whose path matches filter, returning how many changed. Buffers (quantized weights) and integer parameters stay non-trainable.
Source§

fn train(&mut self, on: bool)

Sets every mode flag of the tree: training (true, the default) or evaluation (false, where dropout is the identity), as MLX’s Module.train(mode). (There is no eval(): it would read like x.eval(), which evaluates.)
Source§

fn training(&self) -> bool

Whether the tree is in training mode (its first mode flag; true without any).
Source§

fn load_weights(&mut self, src: impl WeightSource) -> Result<()>

Replaces leaves by name, strictly (MLX’s load_weights(strict=True)): every leaf must be given with its shape and nothing else (loaded dtypes are kept). On any problem the error lists every missing, unknown and mismatched name, and the model is unchanged. src is a safetensors path or named arrays.
Source§

fn load_weights_partial(&mut self, src: impl WeightSource) -> Result<()>

Replaces the leaves given by name (MLX’s load_weights(strict=False)): unknown names are ignored, missing leaves keep their values, shapes must match.
Source§

fn save_weights(&self, path: impl AsRef<Path>) -> Result<()>

Writes every leaf as safetensors with MLX’s names and dtypes (MLX’s save_weights), evaluating them. Unloaded placeholders are an error naming them.
Source§

fn save_trainable(&self, path: impl AsRef<Path>) -> Result<()>

Writes only the trainable leaves (an adapter’s weights, say).
Source§

fn map_modules<T: Tree>( &mut self, filter: impl Fn(&str) -> bool, f: impl FnMut(&str, &mut T) -> Result<()>, ) -> Result<usize>

Calls f(path, submodule) on every submodule of type T whose path matches filter (f may replace it, as when applying LoRA), returning how many. It stops at the first error from f; submodules changed before it keep their change. Matching submodules are not searched for further matches.
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.