Struct MmapFileInner

Source

pub struct MmapFileInner { /* private fields */ }

Expand description

High-performance memory-mapped file (Unsafe lock-free version)

基于内存映射的高性能文件（Unsafe 无锁版本）

⚠️ Warning: This is an unsafe version. Users must ensure that concurrent writes do not overlap.

⚠️ 警告：这是 unsafe 版本，用户需自行保证并发写入的范围不重叠。

It is recommended to use MmapFile + RangeAllocator for compile-time safety.

推荐使用 MmapFile + RangeAllocator 实现编译期安全。

This file implementation is optimized for concurrent random write scenarios.

专为并发随机写入场景优化的文件实现。

§Features

Zero-copy writes: Write operations directly modify mapped memory without system calls
Lock-free concurrency: Concurrent writes to different regions require no locking for maximum performance
Reference counting: Can be cloned and shared among multiple workers
Manual flushing: Control when data is synchronized to disk for optimized batch operations
Runtime agnostic: Does not depend on any specific async runtime

§特性

零拷贝写入：写入操作直接修改映射内存，无需系统调用
无锁并发：不同区域的并发写入无需加锁，极致性能
引用计数：可以克隆并在多个 worker 间共享
手动刷盘：控制何时将数据同步到磁盘，优化批量操作
运行时无关：不依赖特定异步运行时，可用于任何场景

§Limitations

File size must be specified at creation and cannot be dynamically expanded
Maximum file size is limited by system virtual memory
⚠️ Users must ensure that concurrent writes do not overlap (runtime responsibility)

§限制

创建时必须指定文件大小，不支持动态扩展
文件大小上限受系统虚拟内存限制
⚠️ 用户需要确保不会并发写入重叠的内存区域（运行时责任）

§Safety Notes

This implementation uses UnsafeCell to allow lock-free concurrent writes. As long as:

Different threads write to non-overlapping memory regions
No reads occur to the same region during writes

It is completely safe. However, these guarantees must be maintained by the user.

§安全性说明

这个实现使用 UnsafeCell 来允许无锁并发写入。只要：

不同线程写入不重叠的内存区域
不在写入同时读取同一区域

那么就是完全安全的。但这些保证需要用户自行维护。

§Examples

let file = MmapFileInner::create(&path, NonZeroU64::new(1024).unwrap())?;

// ⚠️ Users must ensure concurrent writes do not overlap
// ⚠️ 用户需自行保证不会并发写入重叠区域
let file1 = file.clone();
let file2 = file.clone();

std::thread::scope(|s| {
    // Safety: Two threads write to non-overlapping regions [0, 512) and [512, 1024)
    // Safety: 两个线程写入不重叠的区域 [0, 512) 和 [512, 1024)
    s.spawn(|| unsafe { file1.write_at(0, &[1; 512]) });
    s.spawn(|| unsafe { file2.write_at(512, &[2; 512]) });
});

unsafe { file.flush()?; }

Implementations§

Source §

impl MmapFileInner

Source

pub fn create(path: impl AsRef<Path>, size: NonZeroU64) -> Result<Self>

Create a new file and map it to memory

创建新文件并映射到内存

If the file already exists, it will be truncated. The file will be pre-allocated to the specified size.

如果文件已存在会被截断。文件会被预分配到指定大小。

§Parameters

path: File path
size: File size in bytes, must be > 0

§参数

path: 文件路径
size: 文件大小（字节），必须大于 0

§Examples

// Create a 10MB file
// 创建 10MB 的文件
let file = MmapFileInner::create(&path, NonZeroU64::new(10 * 1024 * 1024).unwrap())?;

§Errors

Returns InvalidFileSize error if size is 0
Returns corresponding I/O errors if file creation or memory mapping fails

§Errors

如果 size 为 0，返回 InvalidFileSize 错误
如果无法创建文件或映射内存，返回相应的 I/O 错误

Source

pub fn open(path: impl AsRef<Path>) -> Result<Self>

Open an existing file and map it to memory

打开已存在的文件并映射到内存

The file must already exist and have a size > 0.

文件必须已存在且大小大于 0。

§Parameters

path: File path

§参数

path: 文件路径

§Examples

let file = MmapFileInner::open(&path)?;

Source

pub unsafe fn write_at(&self, offset: u64, data: &[u8]) -> usize

Write data at the specified position (lock-free operation)

在指定位置写入数据（无锁操作）

This is an extremely fast operation that writes directly to mapped memory without requiring any locks.

这是一个极快的操作，直接写入映射内存，不需要任何锁。

§Safety

The caller must ensure:

Different threads do not write to overlapping memory regions concurrently
No reads occur to the same region during writes

Violating these constraints leads to data races, which is undefined behavior.

§Safety

调用者需要确保：

不同线程不会并发写入重叠的内存区域
不会在写入时读取同一区域

违反这些约束会导致数据竞争，这是未定义行为。

§Parameters

offset: Write position (byte offset from file start)
data: Data to write

§Returns

Number of bytes actually written

§参数

offset: 写入位置（从文件开头的字节偏移）
data: 要写入的数据

§返回值

返回实际写入的字节数

§Examples

let file = MmapFileInner::create(&path, NonZeroU64::new(1024).unwrap())?;

// Concurrent writes to non-overlapping regions using std::thread
// 使用 std::thread 并发写入不重叠区域
let file1 = file.clone();
let file2 = file.clone();

std::thread::scope(|s| {
    // Safety: Two threads write to non-overlapping regions [0, 5) and [100, 105)
    // Safety: 两个线程写入不重叠的区域 [0, 5) 和 [100, 105)
    s.spawn(|| unsafe { file1.write_at(0, b"hello") });
    s.spawn(|| unsafe { file2.write_at(100, b"world") });
});

unsafe { file.flush()?; }

Source

pub unsafe fn write_all_at(&self, offset: u64, data: &[u8])

Write all data at the specified position

在指定位置写入所有数据

This method guarantees that all data is written, or returns an error if insufficient space is available.

这个方法保证写入所有数据，如果空间不足会返回错误。

§Safety

The caller must ensure:

Different threads do not write to overlapping memory regions concurrently
No reads occur to the same region during writes

§Safety

调用者需要确保：

不同线程不会并发写入重叠的内存区域
不会在写入时读取同一区域

§Parameters

offset: Write position
data: Data to write

§参数

offset: 写入位置
data: 要写入的数据

Source

pub unsafe fn read_at(&self, offset: u64, buf: &mut [u8]) -> Result<usize>

Read data at the specified position

在指定位置读取数据

Reads data from the memory mapping into the buffer.

从内存映射中读取数据到缓冲区。

§Safety

The caller must ensure no writes occur to the same region during reads. Concurrent reads are safe, but concurrent read-write to the same region leads to data races.

§Safety

调用者需要确保不会在读取时写入同一区域。并发读取是安全的，但读写同一区域会导致数据竞争。

§Parameters

offset: Read position
buf: Buffer to receive data

§Returns

Number of bytes actually read

§参数

offset: 读取位置
buf: 接收数据的缓冲区

§返回值

返回实际读取的字节数

Source

pub unsafe fn flush(&self) -> Result<()>

Flush data to disk asynchronously

异步刷新数据到磁盘

Initiates an asynchronous flush operation without blocking for completion. The operating system will write data to disk in the background.

发起异步刷新操作，不会阻塞等待完成。操作系统会在后台将数据写入磁盘。

§Safety

During the flush, the caller must ensure no other threads are modifying the mapped memory. While flush itself is a safe operation, it is marked unsafe for API consistency as it operates on data modified through unsafe methods.

§Safety

在刷新期间，调用者需要确保没有其他线程正在修改映射的内存。虽然 flush 本身是安全的操作，但为了保持 API 一致性，它被标记为 unsafe，因为它操作的是通过 unsafe 方法修改的数据。

§Examples

let file = MmapFileInner::create(&path, NonZeroU64::new(1024).unwrap())?;
unsafe {
    file.write_all_at(0, b"important data");
    file.flush()?; // Flush asynchronously to disk
                   // 异步刷新到磁盘
}

Source

pub unsafe fn sync_all(&self) -> Result<()>

Flush data to disk synchronously

同步刷新数据到磁盘

Synchronously flushes data in memory to disk, blocking until completion. This is slower than flush() but guarantees data has been written to disk.

同步将内存中的数据刷新到磁盘，阻塞直到完成。这比 flush() 慢，但保证数据已经写入磁盘。

§Safety

During the flush, the caller must ensure no other threads are modifying the mapped memory. While sync itself is a safe operation, it is marked unsafe for API consistency as it operates on data modified through unsafe methods.

§Safety

在刷新期间，调用者需要确保没有其他线程正在修改映射的内存。虽然 sync 本身是安全的操作，但为了保持 API 一致性，它被标记为 unsafe，因为它操作的是通过 unsafe 方法修改的数据。

§Examples

let file = MmapFileInner::create(&path, NonZeroU64::new(1024).unwrap())?;
unsafe {
    file.write_all_at(0, b"critical data");
    file.sync_all()?; // Ensure data is written to disk
                      // 确保数据已写入磁盘
}

Source

pub unsafe fn flush_range(&self, offset: u64, len: usize) -> Result<()>

Flush a specific range to disk

刷新指定区域到磁盘

Flushes only a portion of the file to disk, which can improve performance.

只刷新文件的一部分到磁盘，可以提高性能。

§Safety

During the flush, the caller must ensure no other threads are modifying memory in that region.

§Safety

在刷新期间，调用者需要确保没有其他线程正在修改该区域的内存。

§Parameters

offset: Start position of the flush range
len: Length of the flush range

§参数

offset: 刷新区域的起始位置
len: 刷新区域的长度

Source

pub fn size(&self) -> NonZeroU64

Get file size

获取文件大小

Source

pub unsafe fn fill(&self, byte: u8) -> Result<()>

Fill the entire file with a specified byte

填充整个文件为指定字节

Efficiently fills the entire file with the specified value.

高效地将整个文件填充为指定值。

§Safety

The caller must ensure no other threads are reading or writing any part of the file during the fill. This operation modifies the entire file content.

§Safety

调用者需要确保在填充期间没有其他线程正在读写文件的任何部分。此操作会修改整个文件内容。

§Parameters

byte: Fill byte value

§参数

byte: 填充字节

Source

pub unsafe fn zero(&self) -> Result<()>

Zero out the entire file

清零整个文件

Fills the entire file with 0.

将整个文件填充为 0。

§Safety

The caller must ensure no other threads are reading or writing any part of the file during the zeroing. This operation modifies the entire file content.