Skip to main content

ResumeState

Struct ResumeState 

Source
pub struct ResumeState {
    pub names: Vec<String>,
    pub marks: Vec<String>,
}
Expand description

Writes a CMF file in a single pass, payloads first.

CmfModel::write_ref needs every payload addressable at once, so a converter has to hold the whole encoded model — RAM, or a spill file it then copies into the output. For a 300B-class MoE that is ~120 GB written twice. This writer instead reserves a gap at the head of the file, appends each payload the moment it is encoded, and patches the envelope, header and directory into that gap at the end. The bytes are written once and peak disk cost is the finished file.

The gap is the one thing that can go wrong: the directory is not sized until the last tensor arrives. CmfStreamWriter::finish therefore refuses loudly if the head does not fit rather than truncating it, and CmfStreamWriter::head_reserve_for gives callers a safe estimate. What a resumed writer recovers from its manifest: the tensors already on disk and any milestones the producer noted.

Fields§

§names: Vec<String>§marks: Vec<String>

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more