pub struct UploadStore { /* private fields */ }Expand description
All in-flight uploads for this process.
State lives in memory only. A restart loses sessions and the client starts
over; persisting them would mean reconstructing a partial hash across
processes, which is a durability feature nobody has asked for yet. The
staging files a restart leaves behind are swept on startup
(sweep_orphan_parts).
Invariant: no method ever holds the sessions lock and the claimed
lock at the same time. create takes claimed (in its own block, which
closes before anything else runs) and only later, separately, takes
sessions; cancel, sweep, and take_for_complete take sessions
first and always drop that guard — explicitly, where it is not the last
use in the enclosing statement — before reaching claimed through
release/release_destination. That the two methods’ orderings are
opposite (claimed before sessions in one, sessions before claimed
in the other) would be a textbook two-lock deadlock if either ever held
both at once; because neither does, the orderings never actually nest and
there is nothing to cycle on. Preserving this is what makes sessions
vs. claimed safe to reason about independently of append’s
documented (non-deadlocking) contention with sweep — see append’s
doc comment for that argument. Breaking this invariant — folding a
claimed access inside a still-held sessions guard, or vice versa —
would reintroduce a real deadlock that no existing test would catch.
Implementations§
Source§impl UploadStore
impl UploadStore
pub fn new(chunk_size: usize) -> Self
Sourcepub fn chunk_size(&self) -> usize
pub fn chunk_size(&self) -> usize
The chunk size clients are told to use.
Sourcepub fn staging_dir(root: &FsRoot, dest_abs: &Path) -> PathBuf
pub fn staging_dir(root: &FsRoot, dest_abs: &Path) -> PathBuf
Where staging files live for an upload landing at dest_abs.
Inside a jail that is one directory at the root, as it has always been.
Machine-wide there is no single place it could be: complete publishes
by rename, which is only atomic within a filesystem, so staging has to
sit on the same one as the destination. Windows makes this unavoidable
rather than merely preferable — a staging directory on C: cannot be
renamed onto D: at all.
Taking the destination’s own parent, rather than the volume root, keeps
that guarantee on Unix too, where a mount point below / is a different
filesystem and / is usually not writable by the account running this.
The cost is that machine-wide staging is no longer one enumerable
directory, which is what sweep_orphan_parts needs — see its doc.
Sourcepub fn has_live_part_under(&self, dir: &Path) -> bool
pub fn has_live_part_under(&self, dir: &Path) -> bool
살아있는 세션 중 스테이징 파일이 dir 아래에 있는 것이 하나라도 있는가.
트리 삭제가 진행 중인 업로드를 지우지 않기 위한 조회다. 근거는 세션
목록이지 디스크가 아니다: 이전 실행이 남긴 고아 .part는 아무도
소유하지 않으므로 “살아있음“이 아니고, 그것까지 살아있다고 답하면
스윕이 아직 닿지 않은 트리가 무기한 삭제 불가가 된다.
sessions 락만 잡는다. claimed을 함께 잡으면 이 타입의 락 불변식이
깨진다 — 그 이유는 UploadStore의 doc comment에 있다.
Sourcepub fn create(
&self,
root: &FsRoot,
dest_abs: &Path,
dest_rel: String,
size: u64,
sha256: String,
) -> Result<String, UploadError>
pub fn create( &self, root: &FsRoot, dest_abs: &Path, dest_rel: String, size: u64, sha256: String, ) -> Result<String, UploadError>
Open a session for dest_rel, which need not exist yet.
Does not sweep expired sessions itself, even opportunistically — an
earlier version did, right here, before anything else ran. That
silently discarded whatever session the sweep reclaimed: UploadStore
has no AuditSink to record with, so a sweep run from inside this
method structurally cannot leave a trail. The caller
(create_upload_blocking, src/api/fs.rs) now sweeps via the
audit-aware sweep_expired_uploads immediately before calling this,
preserving the ordering the old internal call existed for: reclaim
stale capacity before the cap check below runs, so a session old
enough to matter is freed the moment somebody next asks for a new one
— the same guarantee, just recorded now instead of silent.
Sourcepub fn append(
&self,
id: &str,
offset: u64,
bytes: &[u8],
) -> Result<u64, UploadError>
pub fn append( &self, id: &str, offset: u64, bytes: &[u8], ) -> Result<u64, UploadError>
Append one chunk, returning the offset to send next.
The offset is checked rather than trusted: a retried request that already landed would otherwise be written twice and corrupt the hash.
This holds the session’s own Mutex (and the store-wide sessions
RwLock in its shared read mode) for the full duration of the
seek+write_all below, which is blocking disk I/O — not merely an
in-memory update. That means a concurrent sweep (which needs
sessions’ write lock) blocks until this call finishes, and so does
any other caller of append/take_for_complete/cancel for this same
session id (nothing else can, since only one request should ever be
live per session anyway). Concurrent append calls for different
sessions are unaffected — RwLock read access is shared. Accepted:
bounded by one write’s duration. This is not the only possible
shape, though: HashMap<String, Arc<Mutex<Session>>> would let this
function clone the Arc, drop the sessions read guard immediately,
and hold only the session’s own Mutex across the write — removing
the contention with sweep/create entirely while keeping the same
per-session serialization (two chunks for the same session still
cannot interleave, since the session Mutex alone already prevents
that). That is a real improvement, tracked separately rather than
made here: it restructures the state machine’s storage at the tail
of an already-large task, and the contention it would remove is
bounded and documented, not a correctness gap.
One failure mode this contention analysis does not cover: if a writer
ever panicked while holding sessions’ write guard (create’s
insert, take_for_complete’s or sweep’s remove), std::sync::RwLock
poisons permanently — every later .read() and .write() on it,
including this method’s and sweep’s own, then fails identically and
forever, not “eventually swept once the panic clears.” Unreachable
today, specifically because every write-lock section in this file is a
plain HashMap insert or remove with no disk I/O and nothing else
that can panic — not because poisoning itself is impossible. That is
the condition this note depends on, not a permanent property of the
type: if a future change adds a fallible operation (a write, a
panicking conversion, anything that can unwind) inside one of those
three write-lock sections, this analysis no longer holds and the
unreachability claim needs re-checking against whatever was added.
This says nothing about the per-session Mutex acquired below, which
is held across the seek+write_all disk I/O this doc comment
itself describes. It is unreachable for the same underlying reason,
not the same argument: seek and write_all report failure through
Result, propagated with ? rather than unwound, so nothing in that
critical section can panic either.
Sourcepub fn take_for_complete(&self, id: &str) -> Result<FinishedUpload, UploadError>
pub fn take_for_complete(&self, id: &str) -> Result<FinishedUpload, UploadError>
Finish a session: verify the digest and hand back the staging file.
The session is always removed from sessions. The destination’s
claim, however, survives a successful call — see
release_destination’s doc comment for why. A failed checksum is
different: it is terminal (the bytes on disk are known-wrong, and
leaving them resumable would invite a client to retry into the same
wrong result), and terminal means no rename will ever follow, so the
claim is released immediately in that case — there is nothing left for
a caller to finish acting on.
Sourcepub fn release_destination(&self, dest_rel: &str)
pub fn release_destination(&self, dest_rel: &str)
Release a destination’s claim once the caller has finished acting on
the FinishedUpload a prior take_for_complete handed back — after
the rename lands, or after giving up on it (whichever the caller’s
last step was).
Not folded into take_for_complete itself: an earlier version of this
function released the claim there, immediately on success — which
opened a window between “session removed, claim released” and
“staging file renamed into place” where a second create for the same
dest_rel could succeed and start its own rename racing the first’s,
defeating the reason claimed exists at all. Keeping the claim alive
until the caller explicitly releases it closes that window; the caller
(complete_upload in src/api/fs.rs) calls this on every exit path
after take_for_complete succeeds, success or failure of the rename
alike, so the claim is always released exactly once.
Sourcepub fn cancel(&self, id: &str) -> Option<(String, u64)>
pub fn cancel(&self, id: &str) -> Option<(String, u64)>
Discard a session and its staging file.
Returns the destination and bytes received so far when a session
existed to cancel — None means no such session (unknown, already
completed, already cancelled, or already expired). Widened from a
plain bool for the same reason sweep returns
(id, destination, bytes_received) instead of just dropping what it
finds: the caller (cancel_upload in src/api/fs.rs) records a
terminal audit event, and an event that cannot name which file was
cancelled answers only “a session ended”, not “what happened to this
file” — the question an audit trail exists to answer.
Sourcepub fn sweep(&self, ttl: Duration) -> Vec<(String, String, u64)>
pub fn sweep(&self, ttl: Duration) -> Vec<(String, String, u64)>
Drop sessions idle for longer than ttl.
Returns (id, destination, bytes_received) for each, so the caller can
record a terminal audit event. A session that begins and never ends
leaves a trail showing only a beginning, which is not a trail.