Skip to main content

HyperDir

Struct HyperDir 

Source
pub struct HyperDir<'a> { /* private fields */ }

Implementations§

Source§

impl<'a> HyperDir<'a>

Source

pub async fn open( client: Client, file_config: HyperFileConfig, flags: HyperFileFlags, ) -> Result<Self>

Source

pub async fn create( client: Client, file_config: HyperFileConfig, flags: HyperFileFlags, mode: HyperFileMode, ) -> Result<Self>

Source

pub async fn create_with_interceptor( client: Client, file_config: HyperFileConfig, flags: HyperFileFlags, mode: HyperFileMode, interceptor: impl StagingIntercept<S3Staging> + 'static, ) -> Result<Self>

Source

pub async fn stat_fast( client: Client, file_config: HyperFileConfig, ) -> Result<stat>

Source

pub async fn update_stat_fast( client: Client, file_config: HyperFileConfig, stat: &stat, ) -> Result<stat>

Source§

impl<'a> HyperDir<'a>

expose helper fn

Source

pub fn staging_config(&self) -> &StagingConfig

Source

pub fn with_staging_interceptor( &mut self, i: impl StagingIntercept<S3Staging> + 'static, )

Source

pub async fn fs_reachable_meta_ptrs(&self) -> Result<Vec<BlockPtr>>

Every external storage location this directory references — i.e. its bmap’s metadata (B+tree) node locations, which for a directory are the only external storage it occupies (dirents are inline payload in the leaves). The complete reachable set, for a segment reclaimer.

Fail-closed: on Err treat reachability as unknown and reclaim nothing. See HyperDirFile::reachable_meta_ptrs for the full contract.

Source

pub async fn fs_reachable_segments(&self) -> Result<HashSet<SegmentId>>

The staging segments this directory still references (decoded from fs_reachable_meta_ptrs). Same fail-closed contract.

Source§

impl<'a> HyperDir<'a>

Source

pub const DEFAULT_INLINE_MAX: usize

Default max size for a blob-inlined file: files up to this many bytes are stored as a single ULID-versioned object FILE/<uuid>/<ulid>.blob rather than a full hyperfile. Overridable per-directory via the INLINE_MAX_XATTR xattr.

Source

pub const INLINE_MAX_XATTR: &'static str = "hyperdir.inline_max"

Per-directory xattr name carrying a decimal blob-threshold override.

Source

pub async fn fs_create( client: &Client, uri: &str, flags: FileFlags, mode: FileMode, ) -> Result<Self>

Source

pub async fn fs_create_with_interceptor( client: &Client, uri: &str, flags: FileFlags, mode: FileMode, interceptor: impl StagingIntercept<S3Staging> + 'static, ) -> Result<Self>

Source

pub async fn fs_create_default( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, flags: FileFlags, mode: FileMode, ) -> Result<(Self, Uuid)>

Create a child directory under parent_dir_uuid, returning the new directory handle and the UUID allocated for it.

hyperdir owns directory-identity allocation: a fresh v4 UUID is generated here and the directory’s hyperfile is created at layout.dir_uri(bucket, new_uuid). A ScatterFirstInterceptor is installed toward the parent so the first inode flush commits a scatter into the parent’s !/ namespace. The parent must already exist.

The root directory has no parent; create it with HyperDir::create directly (no interceptor) at layout.root_dir_uri(bucket).

Source

pub async fn fs_create_file( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, flags: FileFlags, mode: FileMode, ) -> Result<(HyperFileHandle<'static>, Uuid)>

Create a regular file name under parent_dir_uuid, returning an open hyperfile handle for byte I/O and the UUID allocated for it.

Mirrors [fs_create_default] but creates a FILE (layout.file_uri) and hands back hyperfile’s Hyper: the namespace (name -> uuid) is hyperdir’s via the scatter interceptor, while byte content is hyperfile’s. An initial flush is issued here so the file’s inode object and the parent’s Create scatter both exist before returning — the file is immediately stat-able and visible to read_dir.

Source

pub async fn fs_create_inline( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, inode_raw: &InodeRaw, ) -> Result<Uuid>

Create a fully-inlined small file/symlink: emit a Create scatter into parent carrying inode_raw (which must already have its inline data set via InodeRaw::set_inline), and allocate its identity uuid – without creating any FILE/<uuid> object. The bytes live entirely in the directory namespace (the dirent/scatter), so resolution returns them with no extra GET. Returns the allocated uuid.

Source

pub async fn fs_update_inline( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, uuid: &Uuid, inode_raw: &InodeRaw, ) -> Result<()>

Re-emit an inlined file’s bytes (an Update scatter for an existing inline identity uuid); used when an open inline file is flushed.

Source

pub async fn fs_create_file_at( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, uuid: &Uuid, flags: FileFlags, mode: FileMode, ) -> Result<HyperFileHandle<'static>>

Create a real FILE/<uuid> object at a CALLER-supplied uuid (rather than a fresh one) under parent/name – used to spill an inlined file to segmented storage once it grows past the inline cap, keeping its identity stable. Mirrors [fs_create_file] but with a given uuid.

Source

pub async fn dir_inline_max( client: &Client, layout: &HyperDirLayout, bucket: &str, dir_uuid: &Uuid, ) -> usize

Effective blob threshold for a directory: its hyperdir.inline_max xattr if set and parseable, else DEFAULT_INLINE_MAX. Cheap enough to call at directory open and cache.

Source

pub async fn set_dir_inline_max( client: &Client, layout: &HyperDirLayout, bucket: &str, dir_uuid: &Uuid, max: usize, ) -> Result<()>

Set a directory’s blob threshold override (0 disables blob inlining for new files under it; callers still decide the tier — see the hyperfs requirement doc).

Source

pub async fn fs_create_blob( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, inode_raw: &InodeRaw, bytes: &[u8], ) -> Result<(Uuid, Ulid)>

Create a blob file (80B < size ≤ threshold): allocate a fresh identity uuid + version ulid, write the data-first <ulid>.blob object, then emit the Create commit scatter into parent carrying inode_raw (which must have i_size == bytes.len() and not FLAG_INLINE) followed by the 16-byte version ULID. No FILE/<uuid> hyperfile is created. Returns (uuid, blob_ulid).

Source

pub async fn fs_write_blob( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, uuid: &Uuid, inode_raw: &InodeRaw, bytes: &[u8], ) -> Result<Ulid>

Write a new version of an existing blob file: write a fresh <ulid>.blob and emit an Update scatter refreshing the dirent inode + version pointer. Older versions remain (retention / undelete) until the version GC reclaims them. Returns the new version ulid.

Source

pub async fn fs_update_blob_meta( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, uuid: &Uuid, inode_raw: &InodeRaw, blob_ulid: Ulid, ) -> Result<()>

Update a blob file’s dirent metadata only, keeping its current version pointer: emits an Update scatter carrying inode_raw plus the EXISTING blob_ulid, without PUTting any bytes and without creating a new version. For an attribute-only change (mode/uid/gid/times) on a blob file, so a chmod costs one scatter instead of rewriting the whole blob.

inode_raw must have the same shape [fs_write_blob] requires (i_size = the blob’s byte length, no FLAG_INLINE); only the metadata fields should differ from what the current version was written with.

Source

pub async fn fs_read_blob( client: &Client, layout: &HyperDirLayout, bucket: &str, file_uuid: &Uuid, blob_ulid: Ulid, ) -> Result<Vec<u8>>

Read a blob file’s current version bytes (one GET, no LIST). blob_ulid comes from the resolved dirent (DirFileEntry::blob_ulid).

Source

pub async fn fs_reconcile_blobs( client: &Client, layout: &HyperDirLayout, bucket: &str, file_uuid: &Uuid, keep: Option<Ulid>, ) -> Result<usize>

Reconcile blob versions under FILE/<uuid>/: delete every *.blob whose ULID != keep (all of them if keep is None). Idempotent. Used to drop a now-segmented file’s stray blobs (keep = None, e.g. after a spill) and, by the GC sweep, to prune superseded versions.

Source

pub async fn delete_all_blob_versions( client: &Client, layout: &HyperDirLayout, bucket: &str, file_uuid: &Uuid, ) -> Result<usize>

Delete every *.blob version object under FILE/<uuid>/ (used after a spill). Idempotent. Thin wrapper over [fs_reconcile_blobs] (no keep).

Source

pub async fn fs_gc_blob_versions( client: &Client, layout: &HyperDirLayout, bucket: &str, file_uuid: &Uuid, keep: Ulid, retention: Duration, ) -> Result<usize>

Reclaim superseded versions of a live blob file: deletes every FILE/<uuid>/*.blob older than keep (the current version, from the dirent’s blob_ulid) whose ULID timestamp is older than retention. The current version is always kept. The ULID timestamp is the write time, so no object read is needed. Returns the count deleted.

Source

pub async fn fs_spill_blob( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, uuid: &Uuid, blob_ulid: Ulid, flags: FileFlags, mode: FileMode, ) -> Result<HyperFileHandle<'static>>

Spill a blob file to a segmented hyperfile at the same uuid (grew past threshold / random write / hard link). Reads the current blob bytes, materializes a real FILE/<uuid> hyperfile with those bytes in a single flush (the commit point: the dirent’s blob_ulid is cleared to 0 by the segmented inode’s scatter), then deletes the blob versions. Idempotent / crash-safe: whichever of {blob, hyperfile} the dirent points at wins, the other is reclaimable by GC. Returns the open hyperfile handle.

Source

pub async fn fs_create_root( client: &Client, layout: &HyperDirLayout, bucket: &str, flags: FileFlags, mode: FileMode, ) -> Result<Self>

Create the root directory (DIR/<nil-uuid>).

The root has no parent, so no scatter interceptor is installed: nothing observes the root’s existence through a parent. Errors with AlreadyExists if the root already exists.

Source

pub async fn fs_open_root( client: &Client, layout: &HyperDirLayout, bucket: &str, flags: FileFlags, ) -> Result<Self>

Open the root directory (DIR/<nil-uuid>).

Source

pub async fn fs_open( client: &Client, uri: &str, flags: FileFlags, ) -> Result<Self>

Source

pub async fn fs_open_opt( client: &Client, uri: &str, flags: FileFlags, runtime_config: &HyperFileRuntimeConfig, ) -> Result<Self>

Source

pub async fn fs_open_or_create_with_default_opt( client: &Client, uri: &str, flags: FileFlags, mode: FileMode, ) -> Result<Self>

Delete a child by emitting a tombstone scatter to its parent directory.

This method does not physically delete the child’s S3 prefix. It opens the child briefly to capture its current InodeRaw, builds a tombstone body (TombstoneHeader || InodeRaw), and emits a Delete scatter into the parent directory’s !/ namespace as a conditional PUT (If-None-Match: *). The next compact on the parent removes the directory-entry mapping but keeps the tombstone scatter in S3 as the authoritative deletion record. Physical reclamation of the child prefix is the job of [fs_gc], which honours retention.

parent_dir_uuid / name identify the entry in the parent; child_uuid is the child’s own UUID and child_is_dir selects the DIR/ vs FILE/ namespace for the child’s prefix.

retention:

  • None => retention_until_unix_ms = 0. The next fs_gc may immediately reclaim the child storage. From the user’s perspective this is “delete now (physical reclamation is asynchronous)”.
  • Some(d) => the child storage is preserved for at least d past now. Until the retention expires fs_gc will leave the child prefix alone, which is what makes a future undelete possible.

Create a hard link new_name in parent_dir_uuid to the existing file target_file_uuid.

Bumps the target file’s authoritative link count (stored in the file’s own inode) and adds a directory entry pointing at the same UUID. Hard links to directories are rejected. Fails with AlreadyExists if new_name already exists in the parent.

The link-count bump and the entry insert are not atomic: a crash between them over-counts (storage leak), never under-counts.

Source

pub async fn fs_rmdir( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, name: &str, child_dir_uuid: &Uuid, retention: Option<Duration>, ) -> Result<()>

Remove an empty child directory.

Opens the child directory and verifies it has no entries (merging its bmap with any outstanding scatter via read_dir); returns ErrorKind::DirectoryNotEmpty otherwise. On success it emits a Delete tombstone into the parent exactly like [fs_unlink], so physical reclamation is deferred to [fs_gc] under retention.

The emptiness check and the tombstone are not atomic. The semantics is best-effort: a create into the child that lands in the window between the check and the tombstone is not seen, so rmdir succeeds and that child becomes a nameless orphan – reclaimed by the GC chain (fs_gc removes this directory, then fs_gc_orphans the child), never a permanent leak or a dangling entry. A create visible at check time yields DirectoryNotEmpty. Strict ENOTEMPTY-always would require a per-directory seal checked by every entry-add path (not implemented).

Source

pub async fn fs_reclaim( client: &Client, layout: &HyperDirLayout, bucket: &str, child_uuid: &Uuid, child_is_dir: bool, ) -> Result<()>

Reclaim a child that was displaced by a replace-over-existing rename.

Unlike [fs_unlink], the displaced child has no name left in any directory (its entry was overwritten in place), so it cannot be reclaimed by the name-keyed tombstone/GC path. We reclaim it directly.

This is idempotent so it is safe to run from both the foreground rename and crash recovery: a file is purged only when its stored nlink <= 1 (the displaced name was the last/only one), never by a blind decrement. A still-hard-linked file (nlink > 1) is left in place — its nlink stays stale-high (a future leak on its final unlink, never data loss). Directories (single link) are purged.

Source

pub async fn reclaim_if_unreferenced( client: &Client, layout: &HyperDirLayout, bucket: &str, file_uuid: &Uuid, ) -> Result<Outcome>

Reclaim a file iff it is no longer referenced, using the .unref-marker arithmetic real_nlink = nlink - count(FILE/<uuid>/*.unref) — no directory scan. Done if real <= 0 and the prefix (incl. markers) was purged; Noop if still referenced or already gone. Safe: a missing marker only over-counts real (a leak the authoritative [fs_gc_orphans]/[fs_gc_orphans_from] backstop catches), never under-counts (which would mis-delete a live file). Drives the keeper’s zero-scan orphan path on a FILE/<uuid>/*.unref event.

Source

pub async fn reclaim_displaced( client: &Client, layout: &HyperDirLayout, bucket: &str, displaced: &Uuid, displaced_is_dir: bool, reclaim_key: &str, ) -> Result<Outcome>

Reclaim a child displaced by a replace-over-existing rename, recording a key-visible .unref marker for the lost reference so orphaning is discoverable without a scan. The marker is keyed by reclaim_key’s ULID, so repeated/concurrent calls for the same displacement (foreground and recovery) write it at most once (If-None-Match). Directories (single link) are purged directly. A file with nlink<=1 (the last reference was the displaced one) is purged immediately and needs no marker.

Source

pub async fn fs_emit_reclaim_intent( client: &Client, layout: &HyperDirLayout, bucket: &str, dst_parent: &Uuid, dst_name: &str, displaced: &Uuid, displaced_is_dir: bool, ) -> Result<String>

Durably record (commit point, If-None-Match: *) that displaced was replaced under dst_parent/dst_name by a rename, so a crash before the foreground reclaim doesn’t orphan its storage. Returns the intent key, which the caller deletes once it has reclaimed. Left-behind intents are completed by [fs_recover_renames].

Source

pub async fn fs_delete_intent( client: &Client, bucket: &str, key: &str, ) -> Result<()>

Delete an intent object (best-effort cleanup of a committed reclaim).

Source

pub async fn fs_setxattr( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, name: &str, value: &[u8], ) -> Result<()>

Set an extended attribute (last-write-wins plain PUT). Stored as a sidecar object <child prefix>/_xattr/<base64url(name)>; it lives under the inode’s prefix, so hyperfile’s prefix-wide unlink reclaims it too. Namespace policy (e.g. user.* only) is enforced by the caller.

Source

pub async fn fs_getxattr( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, name: &str, ) -> Result<Option<Vec<u8>>>

Get an extended attribute’s value, or None if it isn’t set.

Source

pub async fn fs_listxattr( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, ) -> Result<Vec<String>>

List the names of all set extended attributes.

Source

pub async fn fs_removexattr( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, name: &str, ) -> Result<bool>

Remove an extended attribute. Returns false if it wasn’t set.

Source

pub async fn has_live_open_lease( client: &Client, layout: &HyperDirLayout, bucket: &str, file_uuid: &Uuid, ttl_secs: u64, ) -> bool

Reclaim physical storage for tombstoned children whose retention has expired.

Lists scatter objects under the parent directory parent_dir_uuid, picks out the Delete (tombstone) scatters, reads each one’s TombstoneHeader, and for every tombstone whose retention_until_unix_ms <= now_ms:

  1. resolves the child URI from the child UUID carried in the scatter and the file/dir mode read from the tombstone’s InodeRaw,
  2. opens the child staging and calls unlink() to remove the child’s prefix in full (inode + segments + scatter folder),
  3. deletes the tombstone scatter object.

Tombstones whose retention has not yet passed are skipped silently; they remain in S3 and a future fs_gc call after expiration will process them. If a child prefix is already gone (e.g. because a previous partial fs_gc run died between the unlink and the tombstone delete), the tombstone scatter is still removed so the directory’s scatter listing converges.

fs_gc is intentionally separate from fs_compact. Compact is expected to run at high frequency; GC walks tombstones, issues Does any mount currently hold file_uuid open (a live open-lease, i.e. _OPEN/<uuid>/<mount-id> whose owning mount’s heartbeat is younger than ttl_secs)? Dead-owner leases are ignored. Fail-safe: returns true on any list/head uncertainty, so GC never reclaims under a possible open fd. Used to defer reclamation of a keep-open file (FLAG_KEEP_OPEN).

Source

pub async fn fs_gc( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_dir_uuid: &Uuid, ) -> Result<GcStats>

physical deletes, and is appropriate for cron/admin cadence.

Source

pub async fn fs_release(&mut self) -> Result<Cno>

Source

pub async fn fs_flush(&mut self) -> Result<Cno>

Source

pub fn fs_getattr(&self) -> Result<stat>

Source

pub async fn fs_getattr_fast(client: &Client, uri: &str) -> Result<stat>

Source

pub async fn fs_chmod(&mut self, mode: mode_t) -> Result<stat>

Source

pub async fn fs_chown(&mut self, uid: uid_t, gid: gid_t) -> Result<stat>

Source

pub async fn fs_setattr(&mut self, stat: &stat) -> Result<stat>

Source

pub async fn fs_setattr_fast( client: &Client, uri: &str, stat: &stat, ) -> Result<stat>

Source

pub async fn fs_read_entry(&self, name: &str) -> Result<DirFileEntry>

Source

pub async fn fs_resolve_entry(&self, name: &str) -> Result<Option<DirFileEntry>>

Scatter-aware single-name resolve (the cheap lookup path): see HyperDirFile::resolve_entry. Returns None if the name is absent.

Source

pub async fn fs_resolve_entry_fast( client: &Client, layout: &HyperDirLayout, bucket: &str, parent_uuid: &Uuid, name: &str, ) -> Result<Option<DirFileEntry>>

Handle-less single-name resolve: same result as [fs_resolve_entry] but without first opening a directory handle. The lookup hot path (one call per path component) thus avoids the open’s redundant inode GET: zero inode GETs when the name still has pending scatter, one on the compacted fallback (vs. two for open-then-resolve).

Source

pub async fn fs_read_dir(&self) -> Result<Vec<DirFileEntry>>

Source

pub async fn fs_compact(&mut self) -> Result<CompactStats>

Source

pub async fn fs_compact_and_rename( &mut self, old_name: &str, new_name: &str, ) -> Result<CompactStats>

Fold pending scatter and rename old_name to new_name in this directory, committed by a SINGLE flush (see HyperDirFile::compact_then_rename), then run compaction phase 2.

A same-directory rename must fold first (rename works on the B-tree and cannot see scatter); doing it as compact-then-rename costs two commits for one logical change. Same semantics as fs_compact followed by fs_rename, with one commit instead of two.

Source

pub async fn fs_rename(&mut self, old_name: &str, new_name: &str) -> Result<()>

Rename an entry within this (already-open, writable) directory.

Same-directory rename keeps the child’s UUID and storage; only this directory’s name->entry mapping changes, committed by a single inode flush (atomic via hyperfile OCC). The destination must not exist (otherwise AlreadyExists); old_name == new_name is a no-op.

Source

pub async fn fs_exchange(&mut self, name_a: &str, name_b: &str) -> Result<()>

Atomically exchange two entries in this directory (RENAME_EXCHANGE): the identities their names map to are swapped, committed by a single inode flush (atomic via hyperfile OCC). Both names must exist (otherwise NotFound); name_a == name_b is a no-op. Same-directory only — cross-directory exchange would need a bidirectional two-phase commit.

Source

pub async fn fs_rename_across( client: &Client, layout: &HyperDirLayout, bucket: &str, src_parent_uuid: &Uuid, src_name: &str, dst_parent_uuid: &Uuid, dst_name: &str, reclaim_key: Option<&str>, ) -> Result<()>

Rename a child across two different directories.

The child keeps its UUID and storage; only the two parents’ mappings change. Because the two parent inode flushes cannot be made atomic together, a rename intent object is written first as the single commit point: once it exists the rename is logically committed and is completed forward (add to destination, remove from source) idempotently, then the intent is deleted. A crash at any point leaves the intent behind for [fs_recover_renames] to finish.

Source must exist (NotFound) and destination must not (AlreadyExists); both are checked before the commit. The pre-checks and the commit are not atomic, so a destination created concurrently between the check and the apply can be overwritten (a higher-layer concern).

Source

pub async fn fs_recover_renames( client: &Client, layout: &HyperDirLayout, bucket: &str, ) -> Result<usize>

Re-drive any rename intents left behind by an interrupted [fs_rename_across] (e.g. a crash between the commit and the intent delete). Each intent’s forward steps are idempotent, so re-applying a fully-completed rename is a no-op.

Source

pub async fn fs_open_dir( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, flags: FileFlags, ) -> Result<Self>

Open a directory by its UUID.

Source

pub async fn recover_intent( client: &Client, layout: &HyperDirLayout, bucket: &str, intent_key: &str, ) -> Result<Outcome>

Recover a single cross-directory rename intent by key — the per-key, event-driven counterpart of [fs_recover_renames]. Idempotent:

  • Noop if the intent is still fresh (younger than TXN_INTENT_STALE_MS): a live rename normally applies and deletes it within milliseconds, so only an abandoned one is recovered here (avoids racing the live applier); the periodic/backstop pass catches a truly crashed rename once its intent goes stale.
  • Noop if the intent object is already gone (applied + deleted).
  • Done if a stale intent was forward-applied and deleted.
Source

pub async fn process_reclaim( client: &Client, layout: &HyperDirLayout, bucket: &str, reclaim_key: &str, ) -> Result<Outcome>

Process a single displaced-child reclaim intent by key — the per-key, event-driven counterpart of the .reclaim branch of [fs_recover_renames]. Idempotent:

  • if the displaced child is still validly named at dst_name, the rename hasn’t completed → Noop (or delete a clearly-stale intent → Done);
  • otherwise the child is orphaned → reclaim it and delete the intent (Done). Noop if the intent object is already gone.

Recover a single hard-link intent by key — the per-key, event-driven counterpart used by both [fs_recover_renames] and the keeper. The intent exists only between fs_link’s nlink bump and the entry commit, so its presence means the bump happened. Idempotent and safe:

  • Noop if the intent is still fresh (younger than TXN_INTENT_STALE_MS) — never races a live fs_link;
  • Noop if the intent object is already gone;
  • if the link’s directory entry was committed (the link actually succeeded, crashing only before the intent delete), just delete the intent (Done) — the bump is correct;
  • otherwise the link was abandoned: record the phantom nlink bump as a .unref marker keyed by the intent’s ULID, then delete the intent (Done). The marker is the same write-once scheme used for displaced hard links (source-(a)), so it is idempotent across concurrent recoverers — real = nlink − count(.unref) then reflects the abandoned reference, and the file is reclaimed when it falls to zero. (A raw decrement could not be used: a recovered _TXN delete is idempotent in S3, so two recoverers would each decrement.)
Source

pub async fn fs_list_dir_uuids( client: &Client, layout: &HyperDirLayout, bucket: &str, ) -> Result<Vec<Uuid>>

Enumerate every directory’s UUID by listing the DIR/ namespace with a / delimiter. Used by a maintenance pass to drive per-directory fs_compact / fs_gc across the whole tree.

Source

pub async fn fs_list_file_uuids( client: &Client, layout: &HyperDirLayout, bucket: &str, ) -> Result<Vec<Uuid>>

Enumerate every file’s UUID by listing the FILE/ namespace.

Source

pub async fn fs_gc_orphans( client: &Client, layout: &HyperDirLayout, bucket: &str, grace: Duration, ) -> Result<usize>

Reclaim orphaned files: those no directory entry references anymore.

The name-keyed tombstone GC ([fs_gc]) can’t see a file that became nameless without a tombstone (e.g. the rare hard-linked child displaced by a replace-over-existing rename, left with a stale-high nlink). This is the backstop: mark every UUID referenced by any directory’s read_dir (scatter-aware, so freshly created entries count), then purge any FILE/<uuid> not in that set whose inode is older than grace, and any EMPTY DIR/<uuid> not in that set (excluding the root) – catching a crashed mkdir’s nameless directory too. The grace window avoids collecting a child whose creation is still in flight. A non-empty orphan directory subtree is left in place (a documented limitation).

Source

pub async fn fs_gc_orphans_from( client: &Client, layout: &HyperDirLayout, bucket: &str, file_uuids: &[Uuid], dir_uuids: &[Uuid], grace: Duration, ) -> Result<usize>

Inventory-fed variant of [fs_gc_orphans] (no LIST of DIR//FILE/): the caller supplies the candidate file_uuids and the complete dir_uuids (the reference universe — a partial list would mis-delete a referenced file). Reads each directory’s entries (scatter-aware) to build the referenced set, then purges any candidate file / empty candidate dir not referenced and older than grace. Fail-closed: if any directory can’t be read (other than being gone, NotFound), the round aborts with that error and reclaims nothing — never deletes a file on an incomplete reference set. The authoritative, directory-content backstop behind the keeper’s .unref fast path.

Source§

impl HyperDir<'_>

Source

pub async fn fs_getlk( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, owner: &str, kind: LockKind, start: u64, end: u64, ) -> Result<Option<Conflict>>

Test for a conflicting lock without acquiring one (FUSE getlk).

Source

pub async fn fs_setlk( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, owner: &str, kind: LockKind, start: u64, end: u64, pid: u32, ttl_ms: u64, ) -> Result<SetLkOutcome>

Acquire / change / release a byte-range lock (FUSE setlk, non-blocking). Returns Granted or Conflict; transient CAS races are retried.

Source

pub async fn fs_unlock_owner( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, owner: &str, ) -> Result<()>

Release every lock held by owner on this file/dir (FUSE release/flush).

Source

pub async fn fs_lock_renew( client: &Client, layout: &HyperDirLayout, bucket: &str, uuid: &Uuid, is_dir: bool, owner: &str, ttl_ms: u64, ) -> Result<()>

Refresh the TTL of every lock held by owner (maintenance renew). A no-op if the owner holds nothing.

Auto Trait Implementations§

§

impl<'a> !Freeze for HyperDir<'a>

§

impl<'a> !RefUnwindSafe for HyperDir<'a>

§

impl<'a> !Send for HyperDir<'a>

§

impl<'a> !Sync for HyperDir<'a>

§

impl<'a> !UnwindSafe for HyperDir<'a>

§

impl<'a> Unpin for HyperDir<'a>

§

impl<'a> UnsafeUnpin for HyperDir<'a>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

Source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> Scope for T

Source§

fn with<F, R>(self, f: F) -> R
where Self: Sized, F: FnOnce(Self) -> R,

Scoped with ownership.
Source§

fn with_ref<F, R>(&self, f: F) -> R
where F: FnOnce(&Self) -> R,

Scoped with reference.
Source§

fn with_mut<F, R>(&mut self, f: F) -> R
where F: FnOnce(&mut Self) -> R,

Scoped with mutable reference.
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more