pub struct AppendRegion<P: Platform> { /* private fields */ }Expand description
A region store-io appends to. Positions are chosen by store-io from an atomic tail, so appends from many threads never overlap and a written block is never rewritten.
Each append occupies whole blocks: the bytes from the end of the data to the next block boundary are zero and are not caller data. Pack small records into one append (or use a batch) to avoid the padding.
Implementations§
Source§impl<P: Platform> AppendRegion<P>
impl<P: Platform> AppendRegion<P>
Sourcepub fn set_force_data_sync(&self, on: bool)
pub fn set_force_data_sync(&self, on: bool)
Makes every barrier on this region (for a slot, every commit; and
Submitter::barrier_all)
issue the OS data sync (fdatasync, FlushFileBuffers), even
on a power-safe store, where writes are durable at completion
and a barrier is otherwise free; receipts then report
data_synced. For data
whose durability should not rest on a power-safe
classification alone: a device trusted as power-safe that
still reports a volatile write cache has that cache flushed.
(On a device with no volatile cache the OS has nothing to send
it: the data is on media already. On Windows every power-safe
write is already followed by a data flush, so forcing adds
one.) It applies to every handle and generation of the region,
but is not stored: set it again after each open.
Sourcepub fn force_data_sync(&self) -> bool
pub fn force_data_sync(&self) -> bool
Whether barriers on this region always flush (see
Self::set_force_data_sync).
Source§impl<P: Platform> AppendRegion<P>
impl<P: Platform> AppendRegion<P>
Sourcepub fn generation(&self) -> Generation
pub fn generation(&self) -> Generation
The generation this handle writes and reads. Recycling starts the next one; embed it in your records to tell generations apart after a crash.
Sourcepub fn recycle(&self) -> Result<Self, Error>
pub fn recycle(&self) -> Result<Self, Error>
Starts a new generation: waits for every operation in flight on this
region, rewrites the region header with the next generation, makes it
durable, and returns a handle to the new generation with its append
position at 0. No other I/O: the old bytes stay on disk, so records
must carry their generation. Every handle, ticket and position of the
old generation is refused from then on (StaleGeneration).
Must not be called from inside this region’s own scan visitor, nor by
a thread whose own crate::Submitter has operations on this region
it has not reaped: recycle waits for them, and only that thread’s
reap or wait finishes them (the same holds for release).
§Errors
NotWritten(StaleGeneration | NotReady) if the region was already
recycled or released; NotWritten(ReadOnly) if the store or the
region is read-only; Io if the store’s queue could not be prepared
(nothing was written; the region stays usable); Poisoned;
DurabilityUnknown.
Sourcepub fn release(self) -> Result<(), Error>
pub fn release(self) -> Result<(), Error>
Releases the region: waits for every operation in flight, records
the release in the region table (durable: from then on the release
stands), marks its header released (durable), and gives its space
back to the file system. Its contents are undefined from then on;
every handle to it is refused (NotReady), and its space is reused
by later provisioning. A header write that fails before it reaches
the device (refused, a queue that could not be prepared, a header
pair that would not encode or plan) still returns success: the space
is free for provisioning, but not given back to the file system. Not
from a thread with unreaped crate::Submitter operations on it
(see Self::recycle).
§Errors
NotWritten(StaleGeneration | NotReady) if the region was already
recycled or released; NotWritten(ReadOnly) if the store or the
region is read-only; on format 2, NotWritten(Frozen) while a table
page is stale or lost, and NotWritten(Overlap) when removing the
entry would change another region’s verdict at the next open;
Poisoned; DurabilityUnknown; or Io if the space could not be
deallocated (the region is released either way). Once the table
records the release, a release fails only on a poisoned store or
with that Io.
Sourcepub fn append(&self, data: &[u8]) -> Result<WriteTicket, Error>
pub fn append(&self, data: &[u8]) -> Result<WriteTicket, Error>
Appends data (any length of at least one byte) and returns a ticket
once the write has completed. The data is not yet durable: pass the
ticket to Self::sync_through, or use Self::append_durable.
The append starts at a block boundary chosen by store-io and is zero-padded to the next one. Data larger than the largest pooled buffer is written in several pieces, kept in flight together.
§Errors
NotWritten (empty data, region full, pool exhausted, read-only, not
positioned), Io if the store’s queue could not be prepared (nothing
was reserved or written), Poisoned, or DurabilityUnknown (the
domain is now poisoned).
Sourcepub fn sync_through(
&self,
ticket: &WriteTicket,
) -> Result<DurableReceipt, Error>
pub fn sync_through( &self, ticket: &WriteTicket, ) -> Result<DurableReceipt, Error>
Makes everything appended up to and including ticket durable.
§Errors
NotWritten(ForeignPosition | StaleGeneration) for a ticket of another
region or generation; Poisoned; DurabilityUnknown if the flush
failed.
Sourcepub fn append_durable(
&self,
data: &[u8],
) -> Result<(RegionPos, DurableReceipt), Error>
pub fn append_durable( &self, data: &[u8], ) -> Result<(RegionPos, DurableReceipt), Error>
Sourcepub fn batch(&self) -> AppendBatch<P>
pub fn batch(&self) -> AppendBatch<P>
A batch: many records packed back to back, written together and made
durable with one barrier. See AppendBatch.
Sourcepub fn read(&self, offset: u64, out: &mut [u8]) -> Result<usize, Error>
pub fn read(&self, offset: u64, out: &mut [u8]) -> Result<usize, Error>
Reads raw bytes at a block-aligned offset into out (any length).
Returns the bytes read, fewer than out.len() only at the end of the
region’s data area.
The bytes are read into store-io’s own aligned buffers and copied
into out: direct I/O needs aligned memory, and a read whose wait
fails stays in flight, so it must never target the caller’s memory.
To read with no copy, use Self::scan (it lends store-io’s
buffers) or a crate::Submitter’s read_append (it reads into a
buffer taken from the store).
§Errors
NotWritten for a misaligned or out-of-range read;
Corruption(MediaError) if the device reports an unreadable range,
with the exact range in the error’s context; Io with the raw code
for any other failure.
Sourcepub fn scan(
&self,
from: u64,
visit: impl FnMut(ScanItem<'_>) -> ControlFlow<()>,
) -> Result<ScanSummary, Error>
pub fn scan( &self, from: u64, visit: impl FnMut(ScanItem<'_>) -> ControlFlow<()>, ) -> Result<ScanSummary, Error>
Reads the region in order from the block-aligned offset from to the
end of its data area, handing every piece to visit; return
ControlFlow::Break to stop (typically where your valid data ends).
The scan reads with direct I/O and several large reads in flight. It
never stops at bad data: a range the device cannot read is narrowed
to whole sectors (the device’s logical blocks where the probe
reported the direct-I/O alignment, else store blocks), reported as
ScanItem::Unreadable, and skipped.
Scans work on read-only and poisoned stores.
visit may call the store (read, write, provision): it never runs
while the scan holds a queue another call waits for (on a shared
queue the scan puts it back before delivering). This region’s
recycle and release wait for the scan itself to end, so they
must not be called from visit; nor may an append wait there for
room that only the same thread’s crate::Submitter can free.
§Errors
NotWritten(Misaligned | OutOfBounds | PoolExhausted) before any I/O;
Io if a read fails for a reason other than the media.
Sourcepub fn resume_at(&self, offset: u64) -> Result<u64, Error>
pub fn resume_at(&self, offset: u64) -> Result<u64, Error>
Moves the append position forward to offset (rounded up to a block),
typically after recovery found where valid data ends. Never moves
backwards, and only while no append is in flight.
The skipped range counts as durable: a writable open flushed the
device before reading anything, so whatever recovery found is on
media. Bytes appended in this open and not yet made durable never do:
then the durable point stays where it is, and the next sync that
flushes covers both (durable_through reads lower until then).
A read-only open flushes nothing, so there resume_at is refused
(D-072); appends are refused there anyway, and so is a poisoned
store.
§Errors
NotWritten(ReadOnly | TooManyInFlight | OutOfBounds), Poisoned.
Sourcepub fn durable_through(&self) -> u64
pub fn durable_through(&self) -> u64
Every byte before this offset is durable.
Sourcepub fn capacity(&self) -> u64
pub fn capacity(&self) -> u64
How far the data area can grow without new space: the region’s capacity in bytes (the sum of its extents).
Sourcepub fn grow(&self, new_size: u64) -> Result<u64, Error>
pub fn grow(&self, new_size: u64) -> Result<u64, Error>
Grows the data area to at least new_size bytes (rounded up to the
block size): the new range is filled, flushed and verified before
one table page write commits the new size (spec §12). Within the
capacity nothing else moves; past it the region gains one extent of
at least 1 MiB and an eighth of its capacity (after its last one
when that space is free), listed in its header before the commit.
Returns the new size; on a writable store a size at most the current
one writes nothing and returns it. Positions and tickets taken
before stay valid.
§Errors
Unsupported in container format 1; NotWritten (TooLarge past
2^32 blocks; ReadOnly; Frozen while a table page is stale or
lost; NotReady if the filled range does not verify;
StaleGeneration for a recycled region; QueueFull); NoSpace
(RegionLimit for a capped region, Extents when its header lists
as many extents as it can hold, Caller past the tag’s cap,
Growth when the device runs out); Io if the range could not be
probed or allocated; Corruption for metadata that does not decode;
Poisoned; DurabilityUnknown (a fill, a flush, the header or the
page write failed after submission: poisoned).
Sourcepub fn is_empty(&self) -> bool
pub fn is_empty(&self) -> bool
Whether the data area is empty (never for a provisioned region).
Sourcepub fn block_size(&self) -> u64
pub fn block_size(&self) -> u64
Block size: appends are padded to it and reads must be aligned to it.