Struct Writer 

pub struct Writer { /* private fields */ }

Writer for creating Burnpack files.

Takes the tensors to write, each carrying bytes that are either already resident or produced on demand (Tensor::deferred). Deferred tensors are drawn one at a time during the write, so a model need not fit in memory to be saved.

Implementations

impl Writer

pub fn new(tensors: Vec<Tensor>) -> Self

Create a new writer

pub fn with_metadata(self, key: &str, value: &str) -> Self

Builder pattern: add metadata and return self

pub fn with_scalar(self, key: &str, value: Scalar) -> Self

Builder pattern: add a typed scalar and return self.

pub fn auto_extension(self, enable: bool) -> Self

Enable or disable automatic extension appending for file writes.

When enabled (the default), write_to_file and write_to_file_in_place append the canonical crate::EXTENSION when the requested path has no extension. When disabled, both methods use the requested path exactly as provided.

pub fn overwrite(self, enable: bool) -> Self

Allow or refuse replacing an existing file on file writes.

When enabled (the default), write_to_file and write_to_file_in_place replace whatever is at the destination. When disabled, both return Error::AlreadyExists instead. The check is not a separate step that another writer could slip past: write_to_file_in_place makes it when it creates the file, and write_to_file when it publishes the finished one (see AtomicFile::commit_new for the one exception, filesystems without hard links).

pub fn size(&self) -> Result<usize, Error>

Calculate the total size needed for the burnpack data.

This is useful when you want to pre-allocate a buffer for write_into(). The size includes padding bytes for both metadata alignment and tensor alignment.

pub fn write_into(self, buffer: &mut [u8]) -> Result<(), Error>

Write burnpack data into a caller-provided buffer.

The buffer must be large enough to hold all data. Use size() to determine the required buffer size. If the buffer is too small, this will return an error.

This allows the caller to control buffer allocation, enabling optimizations like:

• Buffer reuse across multiple writes
• Custom allocators
• Pinned memory for GPU transfers

On failure the buffer’s contents are unspecified: a deferred Tensor produces its bytes during the write, so an entry that fails partway leaves everything before it already copied in. Callers reusing a buffer across writes cannot treat an error as “nothing happened”. write_to_file has no such caveat; write_to_file_in_place has the same one, on the destination itself.

Arguments

• buffer - Mutable slice to write data into. Must be at least size() bytes.

pub fn into_bytes(self) -> Result<Bytes, Error>

Write to a byte buffer (convenience method).

This allocates a buffer internally and writes the burnpack data. For more control over buffer allocation, use size() + write_into().

pub fn write_to_file<P: AsRef<Path>>(self, path: P) -> Result<(), Error>

Write to a file without ever leaving a partial one at path.

By default, the canonical crate::EXTENSION (.bpk) is appended when path has no extension. Use auto_extension(false) to preserve the path.

The container is built in a scratch sibling of path and renamed into place only once every byte is on disk, so path either ends up holding a complete container or is left exactly as it was. A write can fail partway for reasons that have nothing to do with the caller: the disk fills up, a quota is hit, the device reports an I/O error. It can also fail because a deferred tensor’s provider, which runs during the write, returns an error or hands back a different length than it declared. None of these may leave a truncated file where a valid one used to be.

That much holds everywhere, for failure at the process level: a returned error, a panic, the process being killed. The rename is a single call, so it either took effect or it did not, and neither outcome is a partial file.

Power loss is narrower, and Unix-only. There the data is synced before the rename and the parent directory after it, so a crash mid-save leaves the old container and a crash after Ok leaves the new one - never a mixture, and never a lost save. Elsewhere both halves are missing: the directory sync is unavailable, and the replace carries no documented atomicity guarantee (Windows MoveFileEx is not specified as a single metadata transaction when it replaces an existing file). After power loss the destination may hold the old container or the new one, and the finished scratch file may still be beside it.

With overwrite(false) the finished file is hard-linked into place instead of renamed, and Error::AlreadyExists is returned if anything is at path by then; see AtomicFile::commit_new.

Building alongside the destination has four consequences. write_to_file_in_place avoids them, at the cost of the guarantee above:

• The data is fsynced before the rename, so the call does not return until the bytes are durable rather than merely handed to the page cache.
• Overwriting needs room for a second copy. Re-saving a model over itself transiently occupies twice its size, since the old file keeps its blocks until the rename.
• The destination is replaced rather than truncated, so its ownership and hard links do not carry over. Its permission bits do: on Unix an existing regular file’s mode is copied onto the new file before the rename, so a 0600 container stays 0600 rather than reappearing at the process umask. (Windows has no mode to copy.) A symlink at path is replaced by a regular file rather than followed.
• A hard kill (SIGKILL, OOM) skips the cleanup and strands the scratch file. Scratch names are <file_name>.<pid>-<n>.tmp siblings of the resolved path (after any extension is appended), so leftovers are identifiable and safe to delete once no writer is running.

pub fn write_to_file_in_place<P: AsRef<Path>>( self, path: P, ) -> Result<(), Error>

Write directly to a file, replacing its contents in place.

By default, the canonical crate::EXTENSION (.bpk) is appended when path has no extension. Use auto_extension(false) to preserve the path. With overwrite(false), an existing file is refused rather than replaced.

The file is truncated as soon as writing starts, so a failure partway through (a full disk, an I/O error, a deferred tensor’s provider failing) leaves it truncated, destroying whatever container was there before. Under overwrite(false) that partial file is new, and it blocks a retry until removed.

In exchange it skips the costs of write_to_file: no fsync, no transient second copy, and an existing file keeps its inode, so its ownership and hard links survive and a symlink at path is followed rather than replaced. Use it when those matter more than keeping the previous container on failure.