Crate ranged_mmap

Crate ranged_mmap 

Source
Expand description

High-performance lock-free concurrent file writing library

高性能无锁并发文件写入库

This library provides high-performance memory-mapped file implementations optimized for concurrent random writes. It offers both type-safe and unsafe versions to balance safety and performance needs.

本库提供针对并发随机写入优化的高性能内存映射文件实现。 提供类型安全和 unsafe 版本以平衡安全性和性能需求。

§Features

  • Zero-copy writes: Data is written directly to mapped memory
  • Lock-free concurrency: No locks needed for writes to different regions
  • Type-safe API: MmapFile prevents overlapping writes at compile-time
  • High performance: Avoids frequent system calls
  • Runtime agnostic: Works with any async runtime or without one

§特性

  • 零拷贝写入：数据直接写入映射内存
  • 无锁并发：不同区域的写入无需加锁
  • 类型安全 API：MmapFile 在编译期防止重叠写入
  • 高性能：避免频繁的系统调用
  • 运行时无关：可用于任何异步运行时或无运行时环境

§Quick Start

Use MmapFile with [RangeAllocator] for compile-time safety:

§类型安全版本（推荐）

使用 MmapFile 和 [RangeAllocator] 获得编译期安全：

use ranged_mmap::{MmapFile, Result, allocator::ALIGNMENT};

// Create file and allocator (file size should be 4K aligned)
// 创建文件和分配器（文件大小应为4K对齐）
let (file, mut allocator) = MmapFile::create_default(&path, NonZeroU64::new(ALIGNMENT * 2).unwrap())?;

// Allocate ranges (allocations are 4K aligned)
// 分配范围（分配是4K对齐的）
let range1 = allocator.allocate(NonZeroU64::new(ALIGNMENT).unwrap()).unwrap();
let range2 = allocator.allocate(NonZeroU64::new(ALIGNMENT).unwrap()).unwrap();

// Concurrent writes (compile-time safe!)
// 并发写入（编译期安全！）
std::thread::scope(|s| {
    let f1 = file.clone();
    let f2 = file.clone();
    s.spawn(move || f1.write_range(range1, &vec![1u8; ALIGNMENT as usize]));
    s.spawn(move || f2.write_range(range2, &vec![2u8; ALIGNMENT as usize]));
});

unsafe { file.sync_all()?; }

§Unsafe Version (Maximum Performance)

Use MmapFileInner when you can guarantee safety:

§Unsafe 版本（最大性能）

当你能保证安全时使用 MmapFileInner：

use ranged_mmap::{MmapFileInner, Result};

let file = MmapFileInner::create(&path, NonZeroU64::new(1024).unwrap())?;

// ⚠️ You must ensure non-overlapping writes
// ⚠️ 你必须确保写入不重叠
let file1 = file.clone();
let file2 = file.clone();

std::thread::scope(|s| {
    // Safety: Non-overlapping regions
    // Safety: 不重叠的区域
    s.spawn(|| unsafe { file1.write_at(0, &[1; 512]) });
    s.spawn(|| unsafe { file2.write_at(512, &[2; 512]) });
});

unsafe { file.flush()?; }

§Main Types

  • MmapFile: Type-safe memory-mapped file
  • MmapFileInner: Unsafe high-performance memory-mapped file
  • [RangeAllocator]: Allocates non-overlapping file ranges
  • AllocatedRange: Represents an allocated file range
  • WriteReceipt: Proof that a range has been written
  • SplitUpResult: Result of splitting with 4K upper alignment
  • SplitDownResult: Result of splitting with 4K lower alignment

§主要类型

Modules§

allocator
Range allocator implementation

Structs§

AllocatedRange
Allocated file range
MmapFile
Type-safe memory-mapped file
MmapFileInner
High-performance memory-mapped file (Unsafe lock-free version)
WriteReceipt
Write receipt

Enums§

Error
Error type for ranged-mmap operations
SplitDownResult
Result of split_at_align_down
SplitUpResult
Result of split_at_align_up

Type Aliases§

Result
Result type alias using our custom Error type