Crate atomic_write_file

This crate offers functionality to write and overwrite files atomically, that is: without leaving the file in an intermediate state. Either the new contents of the files are written to the filesystem, or the old contents (if any) are preserved.

This crate implements two main structs: AtomicWriteFile and OpenOptions, which mimic the standard std::fs::File and std::fs::OpenOptions as much as possible.

This crate supports all major platforms, including: Unix systems, Windows, and WASI.

Motivation and Example

Consider the following snippet of code to write a configuration file in JSON format:

use std::io::Write;
use std::fs::File;

let mut file = File::options()
                    .write(true)
                    .create(true)
                    .open("config.json")?;

writeln!(file, "{{")?;
writeln!(file, "  \"key1\": \"value1\",")?;
writeln!(file, "  \"key2\": \"value2\"")?;
writeln!(file, "}}")?;

This code opens a file named config.json, truncates its contents (if the file already existed), and writes the JSON content line-by-line.

If the code is interrupted before all of the writeln! calls are completed (because of a panic, or a signal is received, or the process is killed, or a filesystem error occurs), then the file will be left in a broken state: it will not contain valid JSON data, and the original contents (if any) will be lost.

AtomicWriteFile solves this problem by placing the new contents into the destination file only after it has been completely written to the filesystem. The snippet above can be rewritten using AtomicWriteFile instead of File as follows:

use std::io::Write;
use atomic_write_file::AtomicWriteFile;

let mut file = AtomicWriteFile::options()
                               .open("config.json")?;

writeln!(file, "{{")?;
writeln!(file, "  \"key1\": \"value1\",")?;
writeln!(file, "  \"key2\": \"value2\"")?;
writeln!(file, "}}")?;

file.commit()?;

Note that this code is almost the same as the original, except that it now uses AtomicWriteFile instead of File and there’s an additional call to commit().

If the code is interrupted early, before the call to commit(), the original file config.json will be left untouched. Only if the new contents are fully written to the filesystem, config.json will get them.

How it works

This crate works by creating a temporary file in the same directory as the destination file, and then replacing the destination file with the temporary file once the new contents are fully written to the filesystem.

On Unix, the implementation is roughly equivalent to this pseudocode:

fd = open("/path/to/directory/.filename.XXXXXX", O_WRONLY | O_CLOEXEC);
/* ... write contents ... */
fsync(fd);
rename("/path/to/directory/.filename.XXXXXX", "/path/to/directory/filename");

Where XXXXXX represents a random suffix. On non-Unix platforms, the implementation is similar and uses the equivalent platform-specific system calls.

On Unix, the actual implementation is more robust and makes use of directory file descriptors (and the system calls openat, linkat, renameat) to make sure that, if the directory is renamed or remounted during the operations, the file still ends up in the original destination directory, and no cross-device writes happen.

Notes and Limitations

• If the path of an AtomicWriteFile is a directory or a file that cannot be removed (due to permissions or special attributes), an error will be produced when the AtomicWriteFile is committed. This is in contrast with the standard File, which would instead produce an error at open() time.

• AtomicWriteFile is designed so that the temporary files it creates are automatically removed if an error (such as a panic) occurs. However, if the process is interrupted abruptly (without unwinding or running destructors), temporary files may be left on the filesystem.

• If the path of an AtomicWriteFile is a symlink to another file, the symlink is replaced, and the target of the original symlink is left untouched. If you intend to modify the file pointed by a symlink at open time, call Path::canonicalize() prior to calling AtomicWriteFile::open() or OpenOptions::open(). In the future, handling of symlinks will be better customizable.

• Because AtomicWriteFile works by creating a temporary file, and then replacing the original file (see “how it works” above), some metadata of the original file may be lost:

  • On Unix, it is possible to preserve permissions and ownership of the original file. However, it is not generally possible to preserve the same owner user/group of the original file unless the process runs as root (or with the CAP_CHOWN capability on Linux). See OpenOptionsExt::try_preserve_owner() for more details on the behavior of open() when ownership cannot be preserved.

  • On non-Unix platform, there is no support for preserving file permissions or ownership. Support may be added in the future.

  • On all platforms, there is no support for preserving timestamps, ACLs (POSIX Access Control Lists), Linux extended attributes (xattrs), or SELinux contexts. Support may be added in the future.

Cargo features

unnamed-tmpfile (Linux only)

As explained in how it works, this crate works by creating a temporary file, which is then renamed at commit time. By default, the temporary file has a path on the filesystem, and as such if a crash occurs, there is a chance that the temporary file may be left on the filesystem.

On Linux, the implementation of this crate can make use of anonymous temporary files (files opened with O_TMPFILE) if supported, and the implementation is roughly equivalent to this pseudocode:

fd = open("/path/to/directory", O_TMPFILE | O_WRONLY | O_CLOEXEC);
/* ... write contents ... */
fsync(fd);
link("/proc/self/fd/$fd", "/path/to/directory/.filename.XXXXXX");
rename("/path/to/directory/.filename.XXXXXX", "/path/to/directory/filename");

This feature has the following limitations:

• The use of anonymous temporary files ensures that, if the process is interrupted abruptly before a commit, the temporary file is automatically cleaned up by the operating system. However, if the process is interrupted during a commit, it’s still possible (although unlikely) that a named temporary file will be left inside the destination directory.

• This feature requires the /proc filesystem to be mounted. This makes AtomicWriteFile with unnamed-tmpfile unsuitable for use in processes that run early at boot.

This feature has no effect on platforms other than Linux.

Modules

unix

Unix-specific extensions to AtomicWriteFile and OpenOptions.

Structs

AtomicWriteFile

A file whose contents become visible to users only after the file is committed.

Directory

A borrowed reference to the directory containing an AtomicWriteFile.

OpenOptions

Options to configure how an AtomicWriteFile is opened.