pub enum Precision {
F32,
Q8,
Fp8,
}Expand description
How TransformerLm::save_bin stores each weight.
Variants§
F32
Raw f32, 4 bytes per weight. Lossless, and the only choice that a
training run can resume from without a visible jump in loss.
Q8
Symmetric int8 with one f32 scale per row: 1 byte per weight plus 4
bytes per row. Lossy, for inference and for shipping a model.
Fp8
FP8 E4M3 with one f32 scale per row, for the block linears of the
FLUX-family transformer (mmdit::Dit) only: its modulations stay int8,
its embeddings f32, and crate::pipeline::ImagePipeline loads the
prompt encoder at Precision::Q8. Other loaders read it as F32
and TransformerLm::save_bin refuses it. On a device without FP8
tensor cores the bytes are widened to BF16 before each GEMM.
Trait Implementations§
impl Copy for Precision
Source§impl<'de> Deserialize<'de> for Precision
impl<'de> Deserialize<'de> for Precision
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Deserialize this value from the given Serde deserializer. Read more
impl Eq for Precision
impl StructuralPartialEq for Precision
Auto Trait Implementations§
impl Forget for Precision
impl Freeze for Precision
impl Move for Precision
impl RefUnwindSafe for Precision
impl Send for Precision
impl Sync for Precision
impl Unpin for Precision
impl UnsafeUnpin for Precision
impl UnwindSafe for Precision
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more