1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
//! EarthSciIO — Rust core (component (a)): URL download + a shared
//! content-addressed cache.
//!
//! This is the **first data-loader machinery in the Rust track**. It implements
//! the language-neutral spec under [`../spec`](https://github.com/earthsci/earthsciio/tree/main/spec):
//!
//! - the **shared cache key** `sha256(resolved_url)` ([`cache_key`]) and the
//! on-disk [`cache-format`](https://github.com/earthsci/earthsciio/blob/main/spec/cache-format.md)
//! (`v1/blobs/<key[:2]>/<key>.<ext>`, `meta/<key>.json`, `locks/`, `tmp/`);
//! - the [`Manifest`] format, byte-identical to the Python writer, so a blob
//! fetched by one language is reused and re-validated by the others;
//! - the **transport** and **store** registries
//! ([`registries`](https://github.com/earthsci/earthsciio/blob/main/spec/registries.md)):
//! the active `http`/`https`, `file`, and `cds` (Copernicus CDS API:
//! submit→poll→download) transports and the `local` store — with the ERA5
//! pressure-level request mapping ([`era5`]) building `cds://` URLs;
//! - `$EARTHSCIDATADIR` resolution ([`data_dir`]), **offline mode**
//! ([`Cache::is_offline`], [`Error::CacheMiss`]), the ETag/checksum/TTL
//! validation ladder ([`validate`]), mirror failover, and the pluggable
//! [`auth`] seam;
//! - the concurrency contract — advisory `flock` + atomic rename — so multiple
//! processes sharing one `/scratch.local` cache download a URL exactly once.
//!
//! Component (b) builds on this core: the [`FormatRegistry`]'s **readers**
//! decode a cached blob into native-grid arrays ([`NetcdfReader`]), and the
//! cadence-aware [`Provider`] drives `materialize`/`refresh`/`refresh_times`/
//! `prefetch` over them — returning **raw** native arrays (remap/regrid stay
//! upstream/downstream).
//!
//! # Example — fetch (or reuse) a blob
//!
//! (The `cfg` guard is what lets these compile as doctests on the wasm32
//! target, where the cache does not exist — see "Two targets" below.)
//!
//! ```no_run
//! # #[cfg(not(target_arch = "wasm32"))]
//! # fn main() -> Result<(), earthsciio::Error> {
//! use earthsciio::{Cache, FetchRequest};
//!
//! let cache = Cache::from_env()?; // $EARTHSCIDATADIR + EARTHSCI_OFFLINE
//! let blob = cache.fetch(&FetchRequest::new("https://data.earthsci.dev/era5/2018/11/20181108.nc")
//! .loader("era5"))?;
//! println!("cached at {} ({} bytes)", blob.path.display(), blob.manifest.bytes);
//! # Ok(()) }
//! # #[cfg(target_arch = "wasm32")]
//! # fn main() {}
//! ```
//!
//! # Example — offline, cache-only (hermetic)
//!
//! ```no_run
//! # #[cfg(not(target_arch = "wasm32"))]
//! # fn main() -> Result<(), earthsciio::Error> {
//! use earthsciio::{Cache, FetchRequest};
//!
//! let cache = Cache::builder().data_dir("conformance/corpus/cache").offline(true).build()?;
//! let blob = cache.fetch(&FetchRequest::new("https://data.earthsci.dev/era5/2018/11/20181108.nc"))?;
//! // A miss raises Error::CacheMiss naming the url + key — never a silent empty.
//! # Ok(()) }
//! # #[cfg(target_arch = "wasm32")]
//! # fn main() {}
//! ```
//!
//! # Two targets, one writer
//!
//! Everything above describes the **native** crate. The crate also builds for
//! `wasm32-unknown-unknown`, where it is deliberately a much smaller thing: the
//! **output half only** — the codec profiles, the Zarr v3 sharded writer, and
//! (feature `opfs`) a store backed by the browser's Origin Private File System.
//!
//! That exists so earthscilab's browser tier writes a run's output with *this*
//! writer rather than a fourth implementation in JavaScript
//! (`docs/output-handling.md` §4.2/§4.6): one writer, two hosts.
//!
//! The cache, the transports, the store registry, the `Provider` cadence path
//! and every format **reader** are native-only, because each is built on
//! something a browser tab does not have — blocking sockets, `flock`, a temp
//! file on the blob filesystem, a memmap. They are `cfg`-ed out rather than
//! stubbed: a wasm build that calls for them fails to compile, which is a much
//! better outcome than one that links and then returns an empty dataset.
//! `Cargo.toml`'s target tables carry the same split at the dependency level.
// `opfs`'s dependencies are declared in the wasm32 target table, so on a native
// build the feature would silently activate a module whose imports do not
// resolve. Say so instead.
compile_error!;
pub use ;
// The output half — available on every target.
pub use ;
pub use ;
pub use ;
pub use ;
pub use ;
pub use ;
pub use ;
pub use ;
pub use ;
pub use ;
/// The 0.1.1 spelling of [`DataSource`]. From `.esm` 1.0.0 the declaration is a
/// `data_sources` entry, not a `data_loaders` one, and its consumers spell it
/// `DataSource`; this alias keeps 0.1.1 source-compatible and is dropped in
/// 0.2.0.
pub type DataLoader = DataSource;
/// The 0.1.1 spelling of [`SourceTemporal`]. See [`DataLoader`].
pub type LoaderTemporal = SourceTemporal;
pub use ;