Skip to main content

AppendRegion

Struct AppendRegion 

Source
pub struct AppendRegion<P: Platform> { /* private fields */ }
Expand description

A region store-io appends to. Positions are chosen by store-io from an atomic tail, so appends from many threads never overlap and a written block is never rewritten.

Each append occupies whole blocks: the bytes from the end of the data to the next block boundary are zero and are not caller data. Pack small records into one append (or use a batch) to avoid the padding.

Implementations§

Source§

impl<P: Platform> AppendRegion<P>
where P::Queue: Send,

Source

pub fn set_force_data_sync(&self, on: bool)

Makes every barrier on this region (for a slot, every commit; and Submitter::barrier_all) issue the OS data sync (fdatasync, FlushFileBuffers), even on a power-safe store, where writes are durable at completion and a barrier is otherwise free; receipts then report data_synced. For data whose durability should not rest on a power-safe classification alone: a device trusted as power-safe that still reports a volatile write cache has that cache flushed. (On a device with no volatile cache the OS has nothing to send it: the data is on media already. On Windows every power-safe write is already followed by a data flush, so forcing adds one.) It applies to every handle and generation of the region, but is not stored: set it again after each open.

Source

pub fn force_data_sync(&self) -> bool

Whether barriers on this region always flush (see Self::set_force_data_sync).

Source§

impl<P: Platform> AppendRegion<P>
where P::Queue: Send,

Source

pub fn generation(&self) -> Generation

The generation this handle writes and reads. Recycling starts the next one; embed it in your records to tell generations apart after a crash.

Source

pub fn recycle(&self) -> Result<Self, Error>

Starts a new generation: waits for every operation in flight on this region, rewrites the region header with the next generation, makes it durable, and returns a handle to the new generation with its append position at 0. No other I/O: the old bytes stay on disk, so records must carry their generation. Every handle, ticket and position of the old generation is refused from then on (StaleGeneration).

Must not be called from inside this region’s own scan visitor, nor by a thread whose own crate::Submitter has operations on this region it has not reaped: recycle waits for them, and only that thread’s reap or wait finishes them (the same holds for release).

§Errors

NotWritten(StaleGeneration | NotReady) if the region was already recycled or released; NotWritten(ReadOnly) if the store or the region is read-only; Io if the store’s queue could not be prepared (nothing was written; the region stays usable); Poisoned; DurabilityUnknown.

Source

pub fn release(self) -> Result<(), Error>

Releases the region: waits for every operation in flight, records the release in the region table (durable: from then on the release stands), marks its header released (durable), and gives its space back to the file system. Its contents are undefined from then on; every handle to it is refused (NotReady), and its space is reused by later provisioning. A header write that fails before it reaches the device (refused, a queue that could not be prepared, a header pair that would not encode or plan) still returns success: the space is free for provisioning, but not given back to the file system. Not from a thread with unreaped crate::Submitter operations on it (see Self::recycle).

§Errors

NotWritten(StaleGeneration | NotReady) if the region was already recycled or released; NotWritten(ReadOnly) if the store or the region is read-only; on format 2, NotWritten(Frozen) while a table page is stale or lost, and NotWritten(Overlap) when removing the entry would change another region’s verdict at the next open; Poisoned; DurabilityUnknown; or Io if the space could not be deallocated (the region is released either way). Once the table records the release, a release fails only on a poisoned store or with that Io.

Source

pub fn append(&self, data: &[u8]) -> Result<WriteTicket, Error>

Appends data (any length of at least one byte) and returns a ticket once the write has completed. The data is not yet durable: pass the ticket to Self::sync_through, or use Self::append_durable.

The append starts at a block boundary chosen by store-io and is zero-padded to the next one. Data larger than the largest pooled buffer is written in several pieces, kept in flight together.

§Errors

NotWritten (empty data, region full, pool exhausted, read-only, not positioned), Io if the store’s queue could not be prepared (nothing was reserved or written), Poisoned, or DurabilityUnknown (the domain is now poisoned).

Source

pub fn sync_through( &self, ticket: &WriteTicket, ) -> Result<DurableReceipt, Error>

Makes everything appended up to and including ticket durable.

§Errors

NotWritten(ForeignPosition | StaleGeneration) for a ticket of another region or generation; Poisoned; DurabilityUnknown if the flush failed.

Source

pub fn append_durable( &self, data: &[u8], ) -> Result<(RegionPos, DurableReceipt), Error>

Appends data and makes it durable before returning.

§Errors

As Self::append and Self::sync_through.

Source

pub fn batch(&self) -> AppendBatch<P>

A batch: many records packed back to back, written together and made durable with one barrier. See AppendBatch.

Source

pub fn read(&self, offset: u64, out: &mut [u8]) -> Result<usize, Error>

Reads raw bytes at a block-aligned offset into out (any length). Returns the bytes read, fewer than out.len() only at the end of the region’s data area.

The bytes are read into store-io’s own aligned buffers and copied into out: direct I/O needs aligned memory, and a read whose wait fails stays in flight, so it must never target the caller’s memory. To read with no copy, use Self::scan (it lends store-io’s buffers) or a crate::Submitter’s read_append (it reads into a buffer taken from the store).

§Errors

NotWritten for a misaligned or out-of-range read; Corruption(MediaError) if the device reports an unreadable range, with the exact range in the error’s context; Io with the raw code for any other failure.

Source

pub fn scan( &self, from: u64, visit: impl FnMut(ScanItem<'_>) -> ControlFlow<()>, ) -> Result<ScanSummary, Error>

Reads the region in order from the block-aligned offset from to the end of its data area, handing every piece to visit; return ControlFlow::Break to stop (typically where your valid data ends).

The scan reads with direct I/O and several large reads in flight. It never stops at bad data: a range the device cannot read is narrowed to whole sectors (the device’s logical blocks where the probe reported the direct-I/O alignment, else store blocks), reported as ScanItem::Unreadable, and skipped. Scans work on read-only and poisoned stores.

visit may call the store (read, write, provision): it never runs while the scan holds a queue another call waits for (on a shared queue the scan puts it back before delivering). This region’s recycle and release wait for the scan itself to end, so they must not be called from visit; nor may an append wait there for room that only the same thread’s crate::Submitter can free.

§Errors

NotWritten(Misaligned | OutOfBounds | PoolExhausted) before any I/O; Io if a read fails for a reason other than the media.

Source

pub fn resume_at(&self, offset: u64) -> Result<u64, Error>

Moves the append position forward to offset (rounded up to a block), typically after recovery found where valid data ends. Never moves backwards, and only while no append is in flight.

The skipped range counts as durable: a writable open flushed the device before reading anything, so whatever recovery found is on media. Bytes appended in this open and not yet made durable never do: then the durable point stays where it is, and the next sync that flushes covers both (durable_through reads lower until then). A read-only open flushes nothing, so there resume_at is refused (D-072); appends are refused there anyway, and so is a poisoned store.

§Errors

NotWritten(ReadOnly | TooManyInFlight | OutOfBounds), Poisoned.

Source

pub fn tail(&self) -> u64

Where the next append will start.

Source

pub fn durable_through(&self) -> u64

Every byte before this offset is durable.

Source

pub fn len(&self) -> u64

Data-area size in bytes.

Source

pub fn capacity(&self) -> u64

How far the data area can grow without new space: the region’s capacity in bytes (the sum of its extents).

Source

pub fn grow(&self, new_size: u64) -> Result<u64, Error>

Grows the data area to at least new_size bytes (rounded up to the block size): the new range is filled, flushed and verified before one table page write commits the new size (spec §12). Within the capacity nothing else moves; past it the region gains one extent of at least 1 MiB and an eighth of its capacity (after its last one when that space is free), listed in its header before the commit. Returns the new size; on a writable store a size at most the current one writes nothing and returns it. Positions and tickets taken before stay valid.

§Errors

Unsupported in container format 1; NotWritten (TooLarge past 2^32 blocks; ReadOnly; Frozen while a table page is stale or lost; NotReady if the filled range does not verify; StaleGeneration for a recycled region; QueueFull); NoSpace (RegionLimit for a capped region, Extents when its header lists as many extents as it can hold, Caller past the tag’s cap, Growth when the device runs out); Io if the range could not be probed or allocated; Corruption for metadata that does not decode; Poisoned; DurabilityUnknown (a fill, a flush, the header or the page write failed after submission: poisoned).

Source

pub fn is_empty(&self) -> bool

Whether the data area is empty (never for a provisioned region).

Source

pub fn block_size(&self) -> u64

Block size: appends are padded to it and reads must be aligned to it.

Trait Implementations§

Source§

impl<P: Platform> Clone for AppendRegion<P>

Source§

fn clone(&self) -> Self

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl<P: Platform> Debug for AppendRegion<P>

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

§

impl<P> !RefUnwindSafe for AppendRegion<P>

§

impl<P> !UnwindSafe for AppendRegion<P>

§

impl<P> Forget for AppendRegion<P>

§

impl<P> Freeze for AppendRegion<P>
where Store<P>: Freeze,

§

impl<P> Move for AppendRegion<P>

§

impl<P> Send for AppendRegion<P>
where Store<P>: Send,

§

impl<P> Sync for AppendRegion<P>
where Store<P>: Sync,

§

impl<P> Unpin for AppendRegion<P>
where Store<P>: Unpin,

§

impl<P> UnsafeUnpin for AppendRegion<P>
where Store<P>: UnsafeUnpin,

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.