sapphire_framework_session/session.rs
1//! One replication session over one stream.
2//!
3//! Both sides run this same function. It is symmetric: each sends what the other lacks and
4//! applies what it is sent, and neither is a client.
5//!
6//! The exchange, in order:
7//!
8//! 1. `Hello` — format, workspace, replica id and the sender's version vector. A format
9//! mismatch, or a different workspace, ends the session with `Refused` before any state
10//! is exchanged.
11//! 2. Every path state the peer's version vector does not cover, in pages of [`PAGE`], each
12//! page preceded by the content it inlines (64 KiB or less, and within the peer's
13//! `max_file_size`), then `Done`.
14//! 3. `Want` for content that was not inlined, answered with the content or `Missing`. Once
15//! the peer's `Done` is in, also `Want` for content earlier sessions never got
16//! ([`Replica::unavailable`]).
17//! 4. `Settled`, once the sender will ask for nothing more.
18//! 5. `commit_session` last. An interrupted session leaves the version vector where it was,
19//! and everything is simply resent next time. A completed one commits even when some
20//! content was `Missing`: the paths are recorded, and step 3 of every later session asks
21//! for their content again, from whichever peer it meets.
22//!
23//! # Locking and giving up
24//!
25//! The replica is locked per step, not across the exchange: to build this side's pages, to
26//! apply each page that arrives (and work out what it still needs), to answer each `Want`,
27//! and to materialise at the end. Content frames take no lock at all — they go straight to
28//! the staging directory — so a scan, or another session, runs while a large file is in
29//! flight. That is safe because `apply` joins path states, which commutes with a scan's own
30//! records, and committing only merges the peer's version vector.
31//!
32//! With one file able to take minutes, the exchange is bounded by silence, not by total
33//! time: it gives up once nothing has been read or written for [`IDLE_TIMEOUT`].
34//!
35//! The exchange is one function, [`initial_exchange`], because two callers start here:
36//! `run_session` finishes a session from what it returns, and `open_live_session` (in
37//! `live.rs`) keeps the stream and applies `Live` pushes on it.
38//!
39//! # Why there are two end markers
40//!
41//! `Done` says "my pages are complete" and is sent as soon as they are queued, so the peer
42//! can start asking for content without waiting for anything. That leaves a race worth
43//! naming: a peer that has just applied a page and is about to send `Want` looks exactly
44//! like a peer that needs nothing. A side that stopped reading at `Done` would therefore
45//! drop the `Want` for every file over [`INLINE_LIMIT`] — and, with it, the file. `Settled`
46//! closes the race. It is sent once the sender's want list can no longer grow (the peer's
47//! pages are in and every hash asked for has been answered), so the receiver knows no `Want`
48//! can follow, and both sides exit only after seeing the peer's `Done` *and* `Settled`.
49//!
50//! Content is addressed by its hash, so a page's inline blobs may travel in front of the page
51//! that references them. They do: a receiver that had already applied the page would
52//! otherwise ask for content that is on the wire, turning every inlined file into a `Want`
53//! and a second copy of its bytes.
54//!
55//! # Writers
56//!
57//! Everything this side sends goes through one task (`writer.rs`), so the read loop never
58//! blocks on a write, and its queues are unbounded, so queueing never waits on that task
59//! either. Two `run_session` calls facing each other would otherwise deadlock the moment
60//! both wrote more than the socket buffer, or filled a queue, while each waited to read.
61//!
62//! # Content
63//!
64//! Content is streamed from the file that holds it and received straight into the staging
65//! directory (`incoming.rs`), so neither side holds a whole file in memory. Applying a page
66//! finds content there, which is why `apply` and `fetch_missing` are given [`NoContent`].
67
68use std::collections::HashSet;
69
70use grain_id::GrainId;
71use sapphire_sync::{
72 Content, ContentHash, ContentSource, PathUpdate, Replica, Report, VersionVector,
73};
74use tokio::io::{AsyncRead, AsyncWrite, ReadHalf};
75use tokio::sync::Mutex;
76use tokio::task::JoinHandle;
77use tokio::time::Instant;
78
79use crate::error::{Error, Result};
80use crate::frame::{Frame, read_frame};
81use crate::incoming::Incoming;
82use crate::message::Message;
83use crate::progress::Progress;
84use crate::writer::{Out, Outbox, spawn_writer};
85use crate::{IDLE_TIMEOUT, INLINE_LIMIT, SESSION_FORMAT_VERSION};
86
87/// How many path states travel in one `Updates` message.
88const PAGE: usize = 256;
89
90/// How long the exchange waits for the peer's `Hello`.
91///
92/// The one wait in a session with nothing behind it. Every later read is answering something
93/// the peer asked for or was told, and a peer that stops mid-exchange ends the read with end
94/// of file. The `Hello` is different: the stream is open, the peer has its end, and it may
95/// simply never speak — a bridge that parked the stream for an app server which never
96/// claimed it, or a peer that went away between the dial and its first frame on a transport
97/// with no half-close to report it with.
98///
99/// Waiting for ever is not an option: one silent peer would pin a session, its stream and
100/// whatever drives it for the rest of the process's life. The same reasoning
101/// [`IDLE_TIMEOUT`] applies to the rest of the exchange.
102///
103/// Generous on purpose, and a bound rather than a pace: a peer's bridge may hold a stream
104/// parked for about a minute while it starts a stopped app server, which is waiting that is
105/// working as intended, and nothing waits this out when the peer answers.
106pub const HELLO_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(90);
107
108/// What a session did.
109#[derive(Clone, Debug, Default)]
110pub struct SessionOutcome {
111 /// Path updates sent.
112 pub sent: usize,
113 /// Path updates received.
114 pub received: usize,
115 /// What applying them did.
116 pub report: Report,
117}
118
119/// The content source a session hands the replica: none.
120///
121/// Received content is already in the staging directory by the time a page is applied,
122/// and the replica looks there first.
123pub(crate) struct NoContent;
124
125impl ContentSource for NoContent {
126 fn locate(&self, _: &ContentHash) -> Option<std::path::PathBuf> {
127 None
128 }
129}
130
131/// Why the read loop stopped.
132pub(crate) enum Stop {
133 /// The peer's `Done` and `Settled` are in and this side wants nothing more.
134 Complete,
135 /// The stream ended; the peer is gone.
136 Gone,
137 /// The peer refused to continue, and why.
138 Refused(String),
139 /// The peer sent something this session does not allow.
140 Protocol(String),
141}
142
143/// Everything the initial exchange left behind, for the caller that decides what happens
144/// next.
145pub(crate) struct Exchange<S> {
146 /// The peer's half of the stream, still to be read.
147 pub(crate) reader: ReadHalf<S>,
148 /// Everything this side sends goes through here, and one task writes it.
149 pub(crate) out: Outbox,
150 /// The task owning the write half.
151 pub(crate) writer: JoinHandle<()>,
152 /// When the session last moved bytes, marked by the writer task and the read loop.
153 pub(crate) progress: Progress,
154 /// Everything the peer said it has, as of its `Hello`.
155 pub(crate) peer_vv: VersionVector,
156 /// Whether the peer's `Hello` said it answers a live push with `Ack`.
157 pub(crate) peer_acks: bool,
158 /// Everything this side had when it worked out what to send: the peer has all of it
159 /// once the exchange completes.
160 pub(crate) sent_vv: VersionVector,
161 /// The content being received, if any, carried on into the live phase.
162 pub(crate) incoming: Incoming,
163 /// What the exchange did.
164 pub(crate) outcome: SessionOutcome,
165 /// How the read loop stopped.
166 pub(crate) stop: Stop,
167}
168
169/// Run a session to completion.
170///
171/// Returns once both sides have said `Done` and `Settled` and every piece of content this
172/// side asked for has been answered — with the bytes, or with `Missing`. A stream that ends
173/// first returns without an error, but commits nothing.
174///
175/// `replica` is locked for each step that touches it, never across a wait on the peer — not
176/// for its `Hello` (issue #163), and not while content is in flight. A caller that used to
177/// pre-lock a `Mutex<Replica>` around the whole call should stop: passing the mutex itself
178/// is what leaves the replica free for a scan or another session meanwhile.
179///
180/// A peer that stops making progress is given up on after [`IDLE_TIMEOUT`] with
181/// [`Error::Protocol`]; nothing bounds the session's total length.
182///
183/// `S` must be `'static` because the write half is moved into a task; a stream owned by the
184/// caller (a socket, a spliced pipe, a duplex) is.
185pub async fn run_session<S>(
186 stream: S,
187 replica: &Mutex<Replica>,
188 workspace_id: GrainId,
189) -> Result<SessionOutcome>
190where
191 S: AsyncRead + AsyncWrite + Send + Unpin + 'static,
192{
193 let Exchange {
194 out,
195 writer,
196 progress,
197 peer_vv,
198 outcome,
199 stop,
200 ..
201 } = initial_exchange(stream, replica, workspace_id).await?;
202 match stop {
203 Stop::Complete => {
204 // The peer is still reading: it exits only once it has seen this side's
205 // `Settled`, which is queued but not yet written. Draining the writer delivers
206 // `Done`, `Settled` and every pending answer.
207 let result = {
208 let mut locked = replica.lock().await;
209 materialise(&mut locked, &peer_vv, outcome)
210 };
211 close(out, writer, &progress, result).await
212 }
213 // The peer is gone or has misbehaved, so there is nothing to deliver and a write
214 // may never complete. Abort: nothing is committed either way, and everything is
215 // resent next session.
216 Stop::Gone => {
217 abort(out, writer);
218 Ok(outcome)
219 }
220 Stop::Refused(why) => {
221 abort(out, writer);
222 Err(Error::Refused(why))
223 }
224 Stop::Protocol(why) => {
225 abort(out, writer);
226 Err(Error::Protocol(why))
227 }
228 }
229}
230
231/// Run the initial exchange: `Hello`, this side's pages, and reading until the peer's
232/// `Done` and `Settled` are in and every hash this side asked for has been answered.
233///
234/// Both session kinds start here. Nothing is committed: the caller decides, so that an
235/// interrupted exchange leaves the version vector where it was. An error means the exchange
236/// never reached its end markers, and `run_session`'s own error mapping is the caller's job.
237///
238/// `replica` is locked per step, never across a wait on the peer: briefly at the very start,
239/// to read the two values this side's own `Hello` needs (issue #163); once the peer's `Hello`
240/// is in, to build this side's pages; then for each page that arrives (`apply` and
241/// `needed`) and each `Want` (`content_path`). Content frames go to staging and take no
242/// lock, so a scan or another session gets through while a large file is in flight.
243///
244/// After the `Hello`, the read loop gives up once nothing has been read or written for
245/// [`IDLE_TIMEOUT`].
246pub(crate) async fn initial_exchange<S>(
247 stream: S,
248 replica: &Mutex<Replica>,
249 workspace_id: GrainId,
250) -> Result<Exchange<S>>
251where
252 S: AsyncRead + AsyncWrite + Send + Unpin + 'static,
253{
254 let (mut reader, writer) = tokio::io::split(stream);
255
256 // The writer task owns the write half, so the read loop is never blocked by a peer that
257 // is slow to read. It marks `progress` as it writes, and the read loop as it reads.
258 let progress = Progress::new();
259 let (out_tx, writer_task) = spawn_writer(writer, progress.clone());
260
261 // 1. Hello, before anything else. Reading `replica_id`/`vv` is instantaneous, so the
262 // lock is released again immediately — nothing below here needs the replica until the
263 // peer's own `Hello` is in.
264 let (replica_id, vv, max_file_size) = {
265 let locked = replica.lock().await;
266 (
267 locked.replica_id(),
268 locked.vv().clone(),
269 locked.config().max_file_size,
270 )
271 };
272 send(
273 &out_tx,
274 Out::Control(Message::Hello {
275 format: SESSION_FORMAT_VERSION,
276 workspace_id,
277 replica_id,
278 vv,
279 max_file_size,
280 acks: true,
281 }),
282 )?;
283
284 // A peer that never speaks is given up on rather than waited out: see [`HELLO_TIMEOUT`].
285 // The queue is dropped rather than drained, because a peer that did not send a `Hello`
286 // cannot be expected to be reading either.
287 let hello = match tokio::time::timeout(HELLO_TIMEOUT, read_frame(&mut reader)).await {
288 // Progress like any other frame read: the idle clock starts here, not at the
289 // dial, so a `Hello` that took its time does not count against `IDLE_TIMEOUT`.
290 Ok(frame) => {
291 progress.touch();
292 frame
293 }
294 Err(_) => {
295 abort(out_tx, writer_task);
296 return Err(Error::Protocol(format!(
297 "the peer sent no Hello within {HELLO_TIMEOUT:?}"
298 )));
299 }
300 };
301
302 let (peer_vv, peer_max, peer_acks) = match hello? {
303 Some(Frame::Control(Message::Hello {
304 format,
305 workspace_id: theirs,
306 vv,
307 max_file_size,
308 acks,
309 ..
310 })) => {
311 if format != SESSION_FORMAT_VERSION {
312 let _ = send(
313 &out_tx,
314 Out::Control(Message::Refused(format!(
315 "session format {format}, expected {SESSION_FORMAT_VERSION}"
316 ))),
317 );
318 return refuse(
319 out_tx,
320 writer_task,
321 Error::VersionMismatch {
322 ours: SESSION_FORMAT_VERSION,
323 theirs: format,
324 },
325 )
326 .await;
327 }
328 if theirs != workspace_id {
329 let _ = send(
330 &out_tx,
331 Out::Control(Message::Refused(format!(
332 "this stream is for workspace {workspace_id}, not {theirs}"
333 ))),
334 );
335 return refuse(
336 out_tx,
337 writer_task,
338 Error::Protocol(format!(
339 "workspace mismatch: the peer is syncing {theirs}, this session is for \
340 {workspace_id}"
341 )),
342 )
343 .await;
344 }
345 (vv, max_file_size, acks)
346 }
347 Some(Frame::Control(Message::Refused(why))) => {
348 return refuse(out_tx, writer_task, Error::Refused(why)).await;
349 }
350 other => {
351 return refuse(
352 out_tx,
353 writer_task,
354 Error::Protocol(format!("expected Hello, got {other:?}")),
355 )
356 .await;
357 }
358 };
359
360 // 2. Build the outgoing side as owned data, then hand it to a task. Computing it up
361 // front, in one short hold of the lock, keeps the read loop free while it is written.
362 let (mut incoming, sent_vv, sent, outgoing) = {
363 let replica = replica.lock().await;
364 let incoming = Incoming::new(replica.stager());
365 let delta = replica
366 .delta_for(&peer_vv)
367 .map_err(|e| Error::Sync(e.to_string()))?;
368 // Small content goes inline, ahead of the page that needs it — but none over the
369 // peer's size limit, which it would refuse. Looked up in one batch: a lookup per
370 // hash would rebuild the store index each time.
371 let inline_max = (INLINE_LIMIT as u64).min(peer_max);
372 let inline = replica
373 .content_paths(&inline_hashes(&delta, inline_max))
374 .unwrap_or_default();
375 let mut outgoing: Vec<Out> = Vec::new();
376 for page in delta.chunks(PAGE) {
377 for hash in inline_hashes(page, inline_max) {
378 if let Some((path, len)) = inline.get(&hash) {
379 outgoing.push(Out::Blob {
380 hash,
381 path: path.clone(),
382 len: *len,
383 });
384 }
385 }
386 outgoing.push(Out::Control(Message::Updates(page.to_vec())));
387 }
388 outgoing.push(Out::Control(Message::Done));
389 (incoming, replica.vv().clone(), delta.len(), outgoing)
390 };
391
392 // 3. Read until the peer is done and settled and every hash we asked for has been
393 // answered.
394 let mut wanted: HashSet<ContentHash> = HashSet::new();
395 // Hashes the peer answered without the content this session; not asked for again.
396 let mut lacked: HashSet<ContentHash> = HashSet::new();
397 let mut report = Report::default();
398 let mut received_count = 0usize;
399 let mut peer_done = false;
400 let mut peer_settled = false;
401 let mut settled = false;
402
403 // Every error out of the loop comes out of this block, so that one place below takes the
404 // writer down with it: the peer cannot be expected to read what is still queued.
405 let looped: Result<Stop> = async {
406 for item in outgoing {
407 send(&out_tx, item)?;
408 }
409 Ok(loop {
410 if peer_done && peer_settled && wanted.is_empty() {
411 break Stop::Complete;
412 }
413 let Some(frame) = read_or_idle(&mut reader, &progress).await? else {
414 break Stop::Gone;
415 };
416 match frame {
417 Frame::Control(Message::Updates(updates)) => {
418 received_count += updates.len();
419 // The join is idempotent, so a redelivered page costs nothing.
420 let need = {
421 let mut replica = replica.lock().await;
422 let page = replica
423 .apply(&updates, &NoContent)
424 .map_err(|e| Error::Sync(e.to_string()))?;
425 merge_report(&mut report, page);
426 needed(&replica, &updates)?
427 };
428 // Ask for content the page needs that this session does not have.
429 for hash in need {
430 if wanted.insert(hash) {
431 send(&out_tx, Out::Control(Message::Want(hash)))?;
432 }
433 }
434 }
435 Frame::Control(Message::Done) => {
436 peer_done = true;
437 // Content an earlier session never got: its paths were committed all the
438 // same, so no page carries them again (#297). Asked for once the peer's
439 // pages and the content they inline are all in, so nothing on the wire is
440 // asked for twice.
441 let again = replica
442 .lock()
443 .await
444 .unavailable()
445 .map_err(|e| Error::Sync(e.to_string()))?;
446 for hash in again {
447 if !lacked.contains(&hash) && wanted.insert(hash) {
448 send(&out_tx, Out::Control(Message::Want(hash)))?;
449 }
450 }
451 }
452 Frame::Control(Message::Settled) => peer_settled = true,
453 Frame::Control(Message::Want(hash)) => {
454 let reply = answer(&*replica.lock().await, hash);
455 send(&out_tx, reply)?;
456 }
457 Frame::Control(Message::Missing(hash)) => {
458 // The peer does not have it either. Another peer may: the path stays
459 // unmaterialised, and later sessions ask for it again.
460 wanted.remove(&hash);
461 lacked.insert(hash);
462 }
463 Frame::Control(Message::Aborted(hash)) => {
464 // The peer's file changed under it: as good as `Missing`.
465 if let Err(e) = incoming.aborted(hash) {
466 break Stop::Protocol(reason(e));
467 }
468 wanted.remove(&hash);
469 lacked.insert(hash);
470 }
471 Frame::Control(Message::Live(_)) => {
472 // A `Live` push travels only after both sides have left the exchange, and
473 // frames from one peer arrive in order, so a compliant peer cannot send one
474 // here.
475 break Stop::Protocol(
476 "a Live push before the initial exchange finished".to_owned(),
477 );
478 }
479 Frame::Control(Message::Ack(_)) => {
480 // An `Ack` answers a `Live` push, so it cannot come before one either.
481 break Stop::Protocol("an Ack before the initial exchange finished".to_owned());
482 }
483 Frame::Control(Message::Refused(why)) => {
484 break Stop::Refused(why);
485 }
486 Frame::Control(Message::Hello { .. }) => {
487 break Stop::Protocol("a second Hello".to_owned());
488 }
489 // Content is verified before it is staged: it is addressed by hash, so a peer that
490 // sent the wrong bytes under one must not be able to plant them.
491 Frame::BlobStart { hash, len } => {
492 if let Err(e) = incoming.start(hash, len) {
493 break Stop::Protocol(reason(e));
494 }
495 }
496 Frame::BlobChunk(bytes) => {
497 if let Err(e) = incoming.chunk(&bytes) {
498 break Stop::Protocol(reason(e));
499 }
500 }
501 Frame::BlobEnd => match incoming.end() {
502 Ok(hash) => {
503 wanted.remove(&hash);
504 }
505 Err(e) => break Stop::Protocol(reason(e)),
506 },
507 }
508 // The peer's pages are all in, so this side's want list can no longer grow: settling
509 // tells the peer it may close as soon as it wants nothing more either.
510 if peer_done && wanted.is_empty() && !settled {
511 settled = true;
512 send(&out_tx, Out::Control(Message::Settled))?;
513 }
514 })
515 }
516 .await;
517 let stop = match looped {
518 Ok(stop) => stop,
519 // The writer failed to write, so the stream is gone: the same end as the read side
520 // seeing it close, whichever of the two notices first.
521 Err(Error::Closed) => Stop::Gone,
522 Err(e) => {
523 abort(out_tx, writer_task);
524 return Err(e);
525 }
526 };
527
528 Ok(Exchange {
529 sent_vv,
530 reader,
531 out: out_tx,
532 writer: writer_task,
533 progress,
534 peer_vv,
535 peer_acks,
536 incoming,
537 outcome: SessionOutcome {
538 sent,
539 received: received_count,
540 report,
541 },
542 stop,
543 })
544}
545
546/// Read the peer's next frame, or fail once the session has made no progress — nothing
547/// read and nothing written — for [`IDLE_TIMEOUT`].
548///
549/// The writer task marks `progress` as it writes, so a side busy sending a large file is
550/// not cut off for hearing nothing back. Each frame read marks it too.
551pub(crate) async fn read_or_idle<R: AsyncRead + Unpin>(
552 r: &mut R,
553 progress: &Progress,
554) -> Result<Option<Frame>> {
555 // `read_frame` is not cancel-safe: dropping it mid-frame would lose the bytes it has
556 // read. One pinned future survives every re-armed deadline below.
557 let read = read_frame(r);
558 tokio::pin!(read);
559 loop {
560 let deadline = Instant::now() + IDLE_TIMEOUT.saturating_sub(progress.idle_for());
561 tokio::select! {
562 frame = &mut read => {
563 progress.touch();
564 return frame;
565 }
566 // The writer may have moved since the deadline was set; only a full
567 // `IDLE_TIMEOUT` of nothing ends the session.
568 _ = tokio::time::sleep_until(deadline) => {
569 if progress.idle_for() >= IDLE_TIMEOUT {
570 return Err(Error::Protocol(format!("no progress for {IDLE_TIMEOUT:?}")));
571 }
572 }
573 }
574 }
575}
576
577/// Materialise what was waiting on content, then commit.
578///
579/// Committing is what makes this replica claim it has what the peer had. It runs last, and
580/// only when the peer said `Done` and `Settled`; an interrupted session leaves the version
581/// vector where it was.
582pub(crate) fn materialise(
583 replica: &mut Replica,
584 peer_vv: &VersionVector,
585 mut outcome: SessionOutcome,
586) -> Result<SessionOutcome> {
587 let fetched = replica
588 .fetch_missing(&NoContent)
589 .map_err(|e| Error::Sync(e.to_string()))?;
590 merge_report(&mut outcome.report, fetched);
591 replica
592 .commit_session(peer_vv)
593 .map_err(|e| Error::Sync(e.to_string()))?;
594 tracing::debug!(
595 sent = outcome.sent,
596 received = outcome.received,
597 changed = outcome.report.changed,
598 "session complete"
599 );
600 Ok(outcome)
601}
602
603/// Queue one frame, or report that the writer is gone.
604pub(crate) fn send(out: &Outbox, item: Out) -> Result<()> {
605 out.send(item)
606}
607
608/// The answer to the peer's `Want(hash)`: the content, streamed from the file that holds
609/// it, or `Missing`.
610pub(crate) fn answer(replica: &Replica, hash: ContentHash) -> Out {
611 match replica.content_path(&hash) {
612 Ok(Some((path, len))) => Out::Blob { hash, path, len },
613 _ => Out::Control(Message::Missing(hash)),
614 }
615}
616
617/// What a [`Stop::Protocol`] says, from the error that stopped the read loop.
618pub(crate) fn reason(err: Error) -> String {
619 match err {
620 Error::Protocol(why) => why,
621 other => other.to_string(),
622 }
623}
624
625/// Close the outgoing side, let the writer flush what is queued, and return `result`.
626///
627/// Dropping the sender is what ends the writer task; it drains first, so a `Refused` or a
628/// `Settled` queued moments ago still reaches the peer before the stream goes away.
629///
630/// The wait is bounded like the exchange: a peer that holds the stream open but stops
631/// reading would otherwise keep the writer, and this call, waiting for ever. Once the
632/// writer has moved nothing for [`IDLE_TIMEOUT`] it is aborted; `result` stands either
633/// way, since the session's work is already done.
634async fn close(
635 out_tx: Outbox,
636 mut writer: JoinHandle<()>,
637 progress: &Progress,
638 result: Result<SessionOutcome>,
639) -> Result<SessionOutcome> {
640 drop(out_tx);
641 loop {
642 let deadline = Instant::now() + IDLE_TIMEOUT.saturating_sub(progress.idle_for());
643 tokio::select! {
644 _ = &mut writer => break,
645 // The writer may have moved since the deadline was set; only a full
646 // `IDLE_TIMEOUT` of nothing gives up on it.
647 _ = tokio::time::sleep_until(deadline) => {
648 if progress.idle_for() >= IDLE_TIMEOUT {
649 tracing::debug!("the peer stopped reading; dropping what was still queued");
650 writer.abort();
651 break;
652 }
653 }
654 }
655 }
656 result
657}
658
659/// Send a `Refused` that is already queued, close the writer, and return `err`.
660///
661/// The exchange's own early exits: the peer is still reading, so what was queued has to
662/// reach it before the stream goes away.
663async fn refuse<S>(out_tx: Outbox, writer: JoinHandle<()>, err: Error) -> Result<Exchange<S>> {
664 drop(out_tx);
665 let _ = writer.await;
666 Err(err)
667}
668
669/// Close the outgoing side without waiting to deliver what is queued.
670///
671/// Used only once the peer can no longer be expected to read, where a write may otherwise
672/// never complete.
673pub(crate) fn abort(out_tx: Outbox, writer: JoinHandle<()>) {
674 drop(out_tx);
675 writer.abort();
676}
677
678/// The hashes of the content in `updates` of `max` bytes or less, to send inline.
679fn inline_hashes(updates: &[PathUpdate], max: u64) -> Vec<ContentHash> {
680 updates
681 .iter()
682 .flat_map(|update| &update.versions)
683 .filter_map(|entry| match entry.content {
684 Content::File { hash, len } if len <= max => Some(hash),
685 _ => None,
686 })
687 .collect()
688}
689
690/// Hashes these updates need that neither a file on disk nor the staging directory holds.
691///
692/// Content over this replica's size limit is never needed: it is recorded but not
693/// written, and the peer's `BLOB_START` for it would be refused as a protocol error.
694pub(crate) fn needed(replica: &Replica, updates: &[PathUpdate]) -> Result<Vec<ContentHash>> {
695 let max = replica.config().max_file_size;
696 let hashes: Vec<ContentHash> = updates
697 .iter()
698 .flat_map(|update| &update.versions)
699 .filter_map(|entry| match entry.content {
700 Content::File { hash, len } if len <= max => Some(hash),
701 _ => None,
702 })
703 .collect();
704 // One batch per page: a lookup per hash would rebuild the store index each time.
705 let have = replica
706 .content_paths(&hashes)
707 .map_err(|e| Error::Sync(e.to_string()))?;
708 Ok(hashes
709 .into_iter()
710 .filter(|hash| !have.contains_key(hash))
711 .collect())
712}
713
714pub(crate) fn merge_report(into: &mut Report, from: Report) {
715 into.changed += from.changed;
716 into.recorded.extend(from.recorded);
717 into.conflicts.extend(from.conflicts);
718 into.skipped.extend(from.skipped);
719}