pub struct Store(/* private fields */);Expand description
A store: one container file in a directory of its own, on one device.
Every method of store_io_engine::Store is available through Deref:
provisioning and looking up regions and slots, reserve, space,
report, poison.
use store_io::{Store, StoreOptions};
let store = Store::create("data/orders", StoreOptions::default())?;
println!("{}", store.report());
drop(store);
let store = Store::open("data/orders")?;
let wal = store.append_region("wal");Implementations§
Source§impl Store
impl Store
Sourcepub fn create(path: impl AsRef<Path>, opts: StoreOptions) -> Result<Self, Error>
pub fn create(path: impl AsRef<Path>, opts: StoreOptions) -> Result<Self, Error>
Creates a new store in path (the directory is created if needed).
The device is probed first: unsafe or unverified devices are refused
unless opts.trust attests or overrides (and then every receipt is
labelled accordingly).
§Errors
AlreadyExists, UnsafeDevice, Unverified, NoSpace, Io.
Sourcepub fn open(path: impl AsRef<Path>) -> Result<Self, Error>
pub fn open(path: impl AsRef<Path>) -> Result<Self, Error>
Opens an existing store for writing, with default options. Takes the
ownership lock, waits out a dead owner’s writes, and verifies every
region against its header. Append regions must be positioned with
resume_at before the first append.
§Errors
NotFound, Locked, UnsafeDevice, Unverified, Corruption,
DurabilityUnknown (a dead owner’s writes could not be made
durable), Io; Unsupported(FormatVersion) for a newer format and for a
store of container format 1 (before 0.5), which opens read-only
(Self::open_readonly) and moves to format 2 by copy_store
(store-io copy).
Sourcepub fn open_readonly(path: impl AsRef<Path>) -> Result<Self, Error>
pub fn open_readonly(path: impl AsRef<Path>) -> Result<Self, Error>
Opens an existing store for reading and recovery only: no write rights, and nothing on disk changes.
§Errors
NotFound, Corruption, Io.
Sourcepub fn open_readonly_with(
path: impl AsRef<Path>,
opts: StoreOptions,
) -> Result<Self, Error>
pub fn open_readonly_with( path: impl AsRef<Path>, opts: StoreOptions, ) -> Result<Self, Error>
Opens an existing store for reading and recovery only, with
explicit options (for example linux_tier: LinuxTier::Sync).
§Errors
Sourcepub fn engine(&self) -> &Store<NativePlatform>
pub fn engine(&self) -> &Store<NativePlatform>
The engine store underneath.
Methods from Deref<Target = Store<NativePlatform>>§
Sourcepub fn export(
&self,
sink: &mut dyn ExportSink,
) -> Result<ExportReport, Box<ExportError>>
pub fn export( &self, sink: &mut dyn ExportSink, ) -> Result<ExportReport, Box<ExportError>>
Exports every region to sink, in the store’s order: see the
module docs and Self::export_with.
§Errors
Sourcepub fn export_with(
&self,
sink: &mut dyn ExportSink,
opts: ExportOptions,
) -> Result<ExportReport, Box<ExportError>>
pub fn export_with( &self, sink: &mut dyn ExportSink, opts: ExportOptions, ) -> Result<ExportReport, Box<ExportError>>
Exports the regions opts names (all by default) to sink: its
begin with the header, then for each region its region, its
bytes through data and its end_region, then its finish.
§Errors
An ExportError (boxed: it holds the report) with how far it got. Before the sink is
called: NotWritten(InvalidOptions) for a region named twice, a
name no region has, a depth out of range, or on a writable open a
region that only a read-only open exports (one of a stale page,
possibly superseded, or at risk: open the store read-only to export
it); NotWritten(NotReady) for an unavailable region or an entry
that does not decode, NotWritten(Frozen) for a stale or lost table
page (both unless ExportOptions::accept_stale_pages);
NotWritten(TooLarge) for a slot object a format-2 slot cannot
hold; NotWritten(StaleGeneration) for a region that changed while
the plan read it; NotWritten(PoolExhausted) with too few buffers
free for the stream; Io if its queue could not be made. At a
region’s turn: NotWritten(StaleGeneration) (recycled) or
NotWritten(NotReady) (released) since the plan or while it
streams, NotWritten(TooLarge) for a slot object committed since
that a format-2 slot cannot hold; a read’s errors as a
ReadStream’s, a media error (Corruption(MediaError), the
range named) among them; NotWritten(TooManyInFlight) if the sink
still holds every lent chunk after settle. Any error of the sink
(in sink).
Sourcepub fn append_region(&self, name: &str) -> Result<AppendRegion<P>, Error>
pub fn append_region(&self, name: &str) -> Result<AppendRegion<P>, Error>
Looks up an existing append region.
§Errors
Error::NotFound if no ready append region has that name.
Sourcepub fn provision_append_region(
&self,
name: &str,
size: u64,
) -> Result<AppendRegion<P>, Error>
pub fn provision_append_region( &self, name: &str, size: u64, ) -> Result<AppendRegion<P>, Error>
Provisions a new append region of at least size bytes.
§Errors
Error::AlreadyExists, Error::NoSpace, or a platform error.
Sourcepub fn provision_append_region_with(
&self,
name: &str,
opts: RegionOptions,
) -> Result<AppendRegion<P>, Error>
pub fn provision_append_region_with( &self, name: &str, opts: RegionOptions, ) -> Result<AppendRegion<P>, Error>
Provisions a new append region as opts say (its size, and in
container format 2 its cap).
§Errors
As Self::provision_append_region; NotWritten(InvalidOptions)
for options RegionOptions::validate refuses; Unsupported for a
cap in container format 1.
Sourcepub fn provision_append_region_default(
&self,
name: &str,
) -> Result<AppendRegion<P>, Error>
pub fn provision_append_region_default( &self, name: &str, ) -> Result<AppendRegion<P>, Error>
Provisions a new append region with the store’s
StoreOptions::region_defaults.
let opts = StoreOptions {
region_defaults: Some(RegionOptions::default().size(1 << 20)),
..StoreOptions::default()
};
let store = Store::create(SimPlatform::new(SimConfig::volatile(1)), Path::new("/db"), opts)?;
let log = store.provision_append_region_default("log")?;
assert_eq!(log.len(), 1 << 20);§Errors
As Self::provision_append_region_with;
NotWritten(InvalidOptions) when the store has no defaults (before
any other refusal).
Sourcepub fn provision_page_region_default(
&self,
name: &str,
) -> Result<PageRegion<P>, Error>
pub fn provision_page_region_default( &self, name: &str, ) -> Result<PageRegion<P>, Error>
Provisions a new page region with the store’s
StoreOptions::region_defaults.
§Errors
Sourcepub fn page_region(&self, name: &str) -> Result<PageRegion<P>, Error>
pub fn page_region(&self, name: &str) -> Result<PageRegion<P>, Error>
Sourcepub fn provision_page_region(
&self,
name: &str,
size: u64,
) -> Result<PageRegion<P>, Error>
pub fn provision_page_region( &self, name: &str, size: u64, ) -> Result<PageRegion<P>, Error>
Provisions a new page region of at least size bytes.
§Errors
Error::AlreadyExists, Error::NoSpace, or a platform error.
Sourcepub fn provision_page_region_with(
&self,
name: &str,
opts: RegionOptions,
) -> Result<PageRegion<P>, Error>
pub fn provision_page_region_with( &self, name: &str, opts: RegionOptions, ) -> Result<PageRegion<P>, Error>
Provisions a new page region as opts say (its size, and in
container format 2 its cap).
§Errors
Sourcepub fn provision_slot(&self, name: &str) -> Result<Slot<P>, Error>
pub fn provision_slot(&self, name: &str) -> Result<Slot<P>, Error>
Provisions a new small-object slot. Its capacity is the block size
minus 176 bytes of headers (Slot::capacity).
§Errors
Error::AlreadyExists, Error::NoSpace, or a platform error.
Releases the table entry of an unavailable region, one that
Self::unavailable_regions lists (or, named by its id
page << 32 | seq, an entry of a kind this version does not know,
report().metadata.invalid_entries: given up deliberately). The
entry’s page is written without it (durable on return); the region’s
bytes are not touched, and its space becomes free where no other
region’s footprint lies. Its name is free again; the repair is
recorded (Self::repairs). Container format 2.
§Errors
NotFound(Region) if id names no unavailable entry (a usable
region is released with release); NotWritten(Overlap), naming a
region whose open-time verdict rests on this entry (spec §9 step 4:
a live region it supersedes, which may read another’s bytes, or an
unavailable one it demoted by an overlap or a name): the entry stays
while that region does (repair a demoted one first; a superseded
one is read-only and cannot be released: the way out is a copy of
the store that leaves unavailable entries out, copy_store’s
accept_stale_pages, v0.5 E6 C4); NotWritten(Frozen) also for a page holding an entry
past its own write that would be kept (repair that one first, when
it can be: one that supersedes a live region cannot, and two on one
page block each other; the way out is then the copy);
NotWritten(Frozen) while
allocation is frozen (a table page is stale or lost: free space
cannot be trusted); NotWritten(ReadOnly); Unsupported(FormatVersion)
for format 1; Poisoned; DurabilityUnknown if the page write
failed after submission (the store is then poisoned; the entry may or
may not be gone at the next open).
Sourcepub fn accept_page_rollback(&self, pages: &[u32]) -> Result<(), Error>
pub fn accept_page_rollback(&self, pages: &[u32]) -> Result<(), Error>
Accepts the rollback of stale table pages (spec §12): for each page
of pages that is Unconfirmed (report().metadata.stale_pages),
its winner is rewritten byte for byte into its failed slot (even an
unreadable one: writing usually remaps the sector) at the next
meta_seq, durably, and read back; once the new copy reads back as
the winner the page counts as Confirmed: whatever its lost commits
held is given up (space they used may be handed out again) and the
extra extents counted allocated for it are free. Every page is
checked before any is written (an error from a check accepts
none). Each accepted page is recorded (Self::repairs); a
Confirmed page is left as it is.
Once no page is stale or lost, allocation is no longer frozen
(Self::frozen) and the regions become writable (not one possibly
superseded, or with a header feature this version may not write).
Not before: a region of an accepted page may be one a lost commit
released, its space then given to a region of another stale page’s
lost commit, whose bytes it would overwrite. The same holds across
a reopen: an accepted page’s two slots then hold the same payload
(as after a heal), and while any page is stale or lost the open
keeps such a page’s regions read-only (also once its older copy is
damaged: any Confirmed page with a damaged older copy counts). A store with a Lost page
stays frozen, the regions of the pages accepted read-only: it is
copied (copy_store). An accepted page’s next_seq counts higher
(raised by the open that found it stale and by each open while its
trace lasts, written by its next commit): the region ids its lost
commit gave out are not given out
again (unless more than 4,096 provisionings failed on that page
between its last two commits). Container format 2.
§Errors
NotWritten(NotReady) for a Lost page, its slots named (no winner
to keep); NotWritten(OutOfBounds) for a page the table does not
have; NotWritten(Frozen) for a page holding an entry whose
footprint_seq is past the page’s own write, the entry named (the
rewrite would change how the next open counts it, and no repair
runs while allocation is frozen: such a store is copied).
Corruption(MediaError) if the page does not read back with both
slots Valid and the same, and Io if it does not read: the write was
durable, so the store is poisoned (what the next open finds is the
page’s state: Confirmed if the new copy reads, else stale, to be
accepted again). NotWritten(ReadOnly); Unsupported(FormatVersion)
for format 1; Poisoned; DurabilityUnknown if the write failed
after submission (the store poisoned; at the next open the page is
still stale, or Confirmed if the new copy reached the media).
Sourcepub fn extent_of(&self, data_size: u64) -> Option<u64>
pub fn extent_of(&self, data_size: u64) -> Option<u64>
Bytes a region of data_size data bytes occupies in the container:
the data size rounded up to the block, plus two header blocks.
Sourcepub fn set_cap(&self, tag: u32, cap: Option<u64>)
pub fn set_cap(&self, tag: u32, cap: Option<u64>)
Sets (or with None removes) the hard cap of tag: the most bytes
its regions and reservations may hold together. Caps live in memory;
set them again after every open.
Sourcepub fn reserve(&self, bytes: u64, tag: u32) -> Result<Reservation<P>, Error>
pub fn reserve(&self, bytes: u64, tag: u32) -> Result<Reservation<P>, Error>
Reserves bytes for later provisioning under tag: checks the tag’s
cap and grows the container so that provisioning from the reservation
cannot run out of space.
§Errors
NoSpace { tag } if the cap or the file system is exceeded (nothing
changes and nothing is poisoned); NotWritten(ReadOnly); Poisoned;
Io for other allocation failures.
Sourcepub fn space(&self) -> SpaceReport
pub fn space(&self) -> SpaceReport
Where space goes: the container, regions, released extents, reservations, and usage per tag.
Sourcepub fn oldest_outstanding(&self) -> Option<Outstanding>
pub fn oldest_outstanding(&self) -> Option<Outstanding>
The oldest operation still in flight on the store’s queues (shared,
parallel and flush) and its submitters, or None when they are all
idle. O(queues + submitters), no I/O; reads the platform’s coarse
clock once. A submitter’s operation counts until its completion is
processed by reap or wait, a read stream’s until next()
collects it.
The age counts from the first call that saw the operation, so a watchdog calling this every second learns of a hung device within a second of its deadline.
Sourcepub fn poisoned(&self) -> Option<FirstCause>
pub fn poisoned(&self) -> Option<FirstCause>
The failure that poisoned the store, if one has: every write and
barrier fails with Poisoned from then on. A cheap check, for a
health probe or a monitor.
Sourcepub fn observe(&self) -> Snapshot
pub fn observe(&self) -> Snapshot
Event counters and sampled per-stage latencies since this handle was
opened. With feature observe off (the default) nothing is recorded
and the snapshot is all zeros.
Sourcepub fn report(&self) -> &StoreReport
pub fn report(&self) -> &StoreReport
What the store learned about its device and what it decided, as
the open found it (never changed after: what repairs change since
is in Self::unavailable_regions, Self::frozen and
Self::repairs).
Sourcepub fn commit_hold(&self) -> CommitHold
pub fn commit_hold(&self) -> CommitHold
Keeps this store’s device group’s commits active until the hold
drops (PLANNING R5.4), so background work on the device (scans,
copies, background Submitters, of this store or another on the
device) stays paced at one piece in flight per call. For a layer
that commits rarely but needs every commit fast: it holds one while
its transactions are open and drops it when idle. The cost: while a
hold lives, background work runs at the paced rate.
Sourcepub fn qos(&self) -> QosReport
pub fn qos(&self) -> QosReport
This store’s device pacing now: whether pacing is on, whether the device’s commits are active, the budget, the background pieces in flight in the device group.
Sourcepub fn io_path(&self) -> IoPath
pub fn io_path(&self) -> IoPath
The kernel interface the store’s I/O uses now, and why: as
StoreReport::io, with every queue made since counted (a ring the
kernel refused for a Submitter or a parallel lane, and what
served instead).
Sourcepub fn poison(&self)
pub fn poison(&self)
Poisons the device domain (for a caller’s watchdog that decided the
device is hung). Every later write and barrier fails with
Error::Poisoned; reads keep working.
Sourcepub fn domain_stats(&self) -> DomainStats
pub fn domain_stats(&self) -> DomainStats
Flush-sharing counters of the device domain.
Sourcepub fn submit_pool_threads(&self) -> Option<usize>
pub fn submit_pool_threads(&self) -> Option<usize>
Worker threads in the submit pool that runs this store’s page batches
and large reads concurrently, once this store’s first such call has
started its parallel queues (the pool may have been started earlier by
another store on the same platform, and stays None if they could
not be started);
None before that, or on a backend that needs no pool (IOCP, the
simulator).
Sourcepub fn regions(&self) -> Vec<(String, RegionKind)>
pub fn regions(&self) -> Vec<(String, RegionKind)>
Names and kinds of the ready regions.
Regions the region table names that open could not use, with the reason (D-090), less any repaired since. Empty for a healthy store. Their names stay reserved.
Sourcepub fn frozen(&self) -> bool
pub fn frozen(&self) -> bool
Whether allocation is frozen now (format 2: a table page is stale
or lost; spec §9 step 2): no provisioning, no growth beyond
capacity, no release, no repair. report().metadata.frozen is what
the open found.
Sourcepub fn repairs(&self) -> Vec<Repair>
pub fn repairs(&self) -> Vec<Repair>
What this handle’s repairs and accepts did, in order
(Self::repair_unavailable, Self::accept_page_rollback): the
store’s log of them, for the caller to record. The library prints
nothing.
Sourcepub fn prefetch(
&self,
ranges: &[StreamRange<'_, P>],
opts: StreamOptions,
) -> Result<ReadStream<P>, Error>
pub fn prefetch( &self, ranges: &[StreamRange<'_, P>], opts: StreamOptions, ) -> Result<ReadStream<P>, Error>
The bytes of several ranges, of this store’s regions, as one
ReadStream on one queue: each range in the order given, each in
offset order (Chunk::range says which; overlapping ranges are
read twice; no range, an empty stream). An append region’s range was
clamped to its durable prefix when it was made. Each region’s gate is
checked again here and as the stream reads it: a retire of a region
ends the stream before any more of that region’s reads are
delivered. An error tied to no read (TooManyInFlight, a failed
wait, Interrupted) names the range being read ahead.
§Errors
NotWritten: ForeignPosition (a range of another store, or of
another open of this one), StaleGeneration or NotReady (a region
recycled or released), PoolExhausted, InvalidOptions; Io if
its queue could not be made.