1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
//! # Local disk cache
//!
//! Where files land ([`BunsenDiskCache`], resolved from options and the
//! environment), digests and links ([`sha256_of`], [`link_or_copy`]), and the
//! [`TransferObserver`] stack every transfer is reported to. All of that is
//! local; reaching the network is the `fetch` feature.
//!
//! The mechanics are the `bunsen-fetch` crate's, which the bundled-asset
//! crates' build scripts fetch with too; this module sorts its failures into
//! bunsen's error kinds and carries the observers on the cache's options.
//!
//! ## Where files land
//!
//! A [`BunsenDiskCache`] has two roots: a cache directory, for files that
//! can be fetched again, and a data directory, for datasets. Its
//! [`BunsenDiskCacheOptions`] may name either one; otherwise
//! [`BUNSEN_CACHE_CONFIG`] resolves them, from `BUNSEN_CACHE_DIR` and
//! `BUNSEN_DATA_DIR` first and then the platform's directories.
//! [`cache_path`](BunsenDiskCache::cache_path) and
//! [`data_path`](BunsenDiskCache::data_path) name a file under either root
//! without touching the disk.
//!
//! The layers above choose the paths. The
//! [`ResourceCache`](crate::data::resources::ResourceCache) keeps
//! digest-pinned files under `<cache>/resources/`, and a
//! [`ShardSet`](crate::data::datasets::ShardSet) keeps a set's shards under
//! `<data>/shards/<name>/`.
//!
//! ## Digests and files
//!
//! SHA-256 over files and streams ([`sha256_of`], [`verify_sha256`],
//! [`HashingReader`]) serves digest-pinned entries. A file whose path
//! carries its digest was verified when it was written, and is trusted on
//! later runs without re-hashing it: the resource cache's
//! [trust model](crate::data::resources::ResourceCache#trust-model).
//!
//! The file helpers name and place files in the cache: the `.partial`
//! beside a destination that a write goes to before it is renamed into
//! place ([`partial_path`]), a link where a copy would do
//! ([`link_or_copy`]), and the file a URL names ([`file_name_from_url`]).
//! None of them reaches the network.
//!
//! ## Transfers
//!
//! A transfer is one file moving into the cache: a download today, and a
//! copy or a verification pass later. Observers watch transfers without
//! taking part in them. The disk cache carries a stack of them
//! ([`BunsenDiskCacheOptions::transfer_observers`]) and tells every
//! observer about every transfer, in registration order. With the
//! `indicatif` feature, the default stack is one progress bar per
//! transfer, drawn on stderr.
//!
//! ## Fetching
//!
//! This module is the one place bunsen reaches the network at run time,
//! and only with the `fetch` feature. Without it nothing here downloads,
//! and a file that only a URL can supply is not found.
pub use *;
pub use *;
pub use *;
pub use *;
pub use *;
pub use *;
pub use *;
pub use *;