git_remote_object_store/object_store/azure.rs
1//! Azure Blob Storage backend for the [`ObjectStore`] trait.
2//!
3//! [`ObjectStore`]: super::ObjectStore
4//!
5//! `AzureStore` wraps `azure_storage_blob`. Like the S3 backend, this
6//! module owns the URL → SDK config translation, the error-code
7//! classifier ([`classify`]), and the credential resolution plumbing.
8//! Unlike S3, the SDK already does parallel range downloads inside
9//! `BlobClient::download()`, so there is no hand-rolled multipart
10//! orchestrator (asymmetric with S3 by design).
11//!
12//! ## Authentication
13//!
14//! The official `azure_storage_blob` 1.0 crate currently exposes only
15//! `Arc<dyn TokenCredential>` (Entra ID) on its constructors. Azurite
16//! does not implement Entra ID without an `--oauth basic` HTTPS setup,
17//! and many production accounts still authenticate with shared keys.
18//! To bridge both, we install our own [`auth::SharedKeySigningPolicy`]
19//! as a per-try [`azure_core::http::policies::Policy`] and pass `None`
20//! for the SDK's `credential` parameter. The SDK then forwards every
21//! request through our policy, which signs the request using the Azure
22//! Storage shared-key v2 scheme. Tracking issue:
23//! `Azure/azure-sdk-for-rust#2975`.
24//!
25//! Resolution order for `?credential=<NAME>` in the URL:
26//!
27//! 1. `AZSTORE_<NAME>_KEY` — base64 account key → shared-key signing.
28//! 2. `AZSTORE_<NAME>_CONNECTION_STRING` — connection string with
29//! `AccountName=` / `AccountKey=` → shared-key signing.
30//! 3. `AZSTORE_<NAME>_SAS` — SAS query string appended verbatim to
31//! every outgoing request URL.
32//!
33//! When no `?credential=` flag is set we fall back to
34//! `azure_identity::DeveloperToolsCredential` (env, workload identity,
35//! managed identity, Azure CLI, ...).
36//!
37//! ## Conditional writes
38//!
39//! [`put_if_absent`][super::ObjectStore::put_if_absent] uses
40//! `If-None-Match: "*"` (the SDK's
41//! `BlockBlobClientUploadOptions::if_not_exists` convenience).
42//! Azure returns 409 (`BlobAlreadyExists`) or 412
43//! (`ConditionNotMet`) for the contention case; both collapse to
44//! `Ok(false)`.
45//!
46//! ## Atomic `get_to_file`
47//!
48//! Identical to the S3 path: `head` → tempfile → `download(if_match)` →
49//! persist. The SDK's `download()` aggregates parallel range fetches
50//! internally, so no per-chunk semaphore here. A single retry with a
51//! fresh `ETag` covers the head-then-`GET` race (412 mid-download).
52//!
53//! ## `copy(src, dst)`
54//!
55//! `azure_storage_blob` 1.0 does not expose a `BlobClient::copy_from_url`
56//! method (only `BlockBlobClient::upload_blob_from_url`, which requires
57//! a SAS-tokened source URL or an `x-ms-copy-source-authorization`
58//! header — neither integrates cleanly with our credential model). We
59//! implement `copy` as a stream-through-tempfile round trip:
60//! `get_to_file` writes `src` to a `NamedTempFile`, then `put_path`
61//! uploads it to `dst`. Both legs already stream — `get_to_file`
62//! consumes the SDK's chunked download into the file without buffering
63//! the body, and `put_path` switches to our explicit
64//! `stage_block` + `commit_block_list` orchestrator (see
65//! [`AzureStore::multipart_put_path`]) once the body crosses
66//! [`super::multipart::MULTIPART_PUT_THRESHOLD`]. Peak in-flight bytes
67//! are bounded by
68//! [`super::multipart::MULTIPART_PUT_MAX_CONCURRENCY`] ×
69//! [`super::multipart::MULTIPART_PUT_PART_SIZE`] regardless of blob
70//! size, which matters for `manage doctor`'s duplicate-bundle
71//! quarantine path ([`crate::manage::doctor::Doctor::evict_losing_bundle`])
72//! — that path can copy multi-GiB bundles. Zero-byte lock files still
73//! round-trip fast: `get_to_file` short-circuits the GET on `size == 0`
74//! and `put_path` issues a single zero-byte `Put Blob`. Body is
75//! preserved; user metadata is not propagated, matching the S3 backend's
76//! `CopyObject` path which similarly carries only body bytes.
77//!
78//! This is asymmetric with the S3 backend, which uses `CopyObject` for
79//! a true server-side copy — Azure's equivalent (`Copy Blob`,
80//! `Put Blob From URL`) requires a SAS-signed source URL or an
81//! `x-ms-copy-source-authorization` header that the 1.0 SDK does not
82//! ergonomically expose. The download+reupload path is the safe
83//! correct fallback until the SDK closes that gap.
84//!
85//! ## A note on `Range` and zero-byte blobs
86//!
87//! A `Range` request against a zero-byte blob returns HTTP 416. We
88//! never issue Range requests directly — `BlobClient::download()`
89//! owns that — but the zero-size short-circuit in
90//! [`get_to_file`](ObjectStore::get_to_file) also avoids any download
91//! SDK call against a known-empty blob, which sidesteps the issue
92//! entirely.
93//!
94//! ## Size limits
95//!
96//! Azure caps a block blob at 50 000 committed blocks (~4.75 TiB at
97//! the SDK's default block size) and a single `Put Blob` body at
98//! 5000 MiB; above [`super::multipart::MULTIPART_PUT_THRESHOLD`] the
99//! helper switches to explicit `stage_block` + `commit_block_list`,
100//! so callers do not have to reason about the single-call cutoff.
101//! The upload path is **not resumable** across process death — see
102//! the README "Known limitations" section.
103//!
104//! ## HTTP transport tuning
105//!
106//! `azure_core` 1.1's default transport keeps idle pooled connections
107//! forever and never sets TCP keepalive, so a pooled connection to a
108//! rotated VIP would hang an in-flight request until the OS-level TCP
109//! retransmit timeout fires (~15 minutes on Linux). [`AzureStore`]
110//! installs a custom [`reqwest::Client`] via [`Transport`] on
111//! [`ClientOptions::transport`] with four bounds:
112//!
113//! - [`POOL_IDLE_TIMEOUT`] (30 s) — drops idle pooled connections
114//! before a typical DNS rotation makes them stale.
115//! - [`TCP_KEEPALIVE`] (30 s) — detects a dead-but-not-closed TCP
116//! session in seconds rather than the 2-hour Linux default; covers
117//! *hot* pooled connections that pool-idle alone cannot.
118//! - [`CONNECT_TIMEOUT`] (10 s) — bounds a fresh-connect attempt to
119//! a dead VIP rather than waiting on the OS connect timeout.
120//! - [`READ_TIMEOUT`] (30 s) — per-read timeout that resets after a
121//! successful read, so a stuck transfer fails fast without limiting
122//! total body size.
123//!
124//! Together these cap a DNS-rotation hang at tens of seconds rather
125//! than minutes. The custom transport leaves
126//! [`ClientOptions::per_try_policies`] (where the shared-key signing
127//! lives) untouched — the SDK pipeline runs per-try policies
128//! independently of the transport. Tracking issue: #26.
129//!
130//! ## Stdout discipline
131//!
132//! Per `.claude/rules/protocol-stdout.md`, this module never writes to
133//! stdout. Diagnostics go through `tracing` (which the helper binaries
134//! configure to write to stderr).
135
136pub mod auth;
137pub(crate) mod sas;
138
139use std::path::Path;
140use std::sync::Arc;
141use std::time::Duration;
142
143use azure_core::http::headers::{HeaderName, Headers};
144use azure_core::http::request::RequestContent;
145use azure_core::http::{ClientOptions, Transport};
146use azure_storage_blob::clients::{
147 BlobClient, BlobContainerClient, BlobContainerClientOptions, BlockBlobClient,
148};
149use azure_storage_blob::models::{
150 BlobClientDeleteOptions, BlobClientDownloadOptions, BlobClientGetPropertiesOptions,
151 BlobContainerClientListBlobsOptions, BlockBlobClientCommitBlockListOptions,
152 BlockBlobClientUploadOptions, BlockLookupList, HttpRange,
153};
154use azure_storage_blob::stream::tokio::FileStream;
155use bytes::Bytes;
156use futures::StreamExt;
157use tempfile::NamedTempFile;
158use time::OffsetDateTime;
159use tokio::io::AsyncWriteExt;
160use tokio::sync::Semaphore;
161use tokio::task::JoinSet;
162use url::Url;
163
164use crate::url::{AzureAddressing, RemoteUrl};
165
166use super::error::{network_boxed, other_boxed};
167use super::multipart::{
168 AZURE_MAX_BLOCKS, MULTIPART_PUT_MAX_CONCURRENCY, MULTIPART_PUT_PART_SIZE, UploadPart,
169 plan_upload_parts, read_file_part, should_use_multipart, slice_bytes_part,
170};
171use super::{
172 GetOpts, ObjectMeta, ObjectStore, ObjectStoreError, ProgressSink, PutOpts, persist_temp,
173};
174
175/// Azure Blob's hard ceiling on a single Put Blob body for the wire
176/// versions we negotiate (2019-12-12+). Reported in
177/// [`ObjectStoreError::PayloadTooLarge`] when the SDK surfaces HTTP 413
178/// or `RequestBodyTooLarge`, so the wire-line names a concrete number
179/// rather than dumping an opaque SDK chain.
180pub(crate) const SINGLE_PUT_BLOB_LIMIT_BYTES: u64 = 5_000 * (1 << 20);
181
182/// Bound on how long an idle pooled HTTPS connection lingers before
183/// the [`reqwest`] connection pool drops it. Short enough that DNS
184/// rotation rarely hits a stale pooled connection; long enough that
185/// bursty fetch / push batches still benefit from connection reuse.
186/// See module-level "HTTP transport tuning" docs and issue #26.
187pub(crate) const POOL_IDLE_TIMEOUT: Duration = Duration::from_secs(30);
188
189/// TCP keepalive interval for the custom [`reqwest`] transport.
190/// Detects dead-but-not-closed sessions in seconds rather than the
191/// 2-hour Linux default. See module-level "HTTP transport tuning"
192/// docs and issue #26.
193pub(crate) const TCP_KEEPALIVE: Duration = Duration::from_secs(30);
194
195/// Bound on a fresh TCP-connect attempt. `reqwest` defaults to no
196/// connect timeout, so an unreachable IP would otherwise wait on the
197/// OS-level connect timeout (~75 s on Linux defaults). 10 s is
198/// comfortable for an in-region or even cross-region handshake while
199/// failing fast on a dead VIP. See module-level "HTTP transport
200/// tuning" docs and issue #26.
201pub(crate) const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
202
203/// Per-read timeout for the custom [`reqwest`] transport. Resets after
204/// each successful read, so it caps how long a stuck connection can
205/// hold a transfer without limiting total body size. Sized to match
206/// [`POOL_IDLE_TIMEOUT`] / [`TCP_KEEPALIVE`] so a single rotation
207/// budget covers all three knobs. See module-level "HTTP transport
208/// tuning" docs and issue #26.
209pub(crate) const READ_TIMEOUT: Duration = Duration::from_secs(30);
210
211/// Production [`ObjectStore`] backed by `azure_storage_blob`.
212pub struct AzureStore {
213 container: BlobContainerClient,
214 /// Container name as parsed from the URL — needed by SAS-token
215 /// construction (issue #76) because the SDK's
216 /// `BlobContainerClient::container_name()` is private. Held
217 /// regardless of credential type so the field shape doesn't
218 /// branch on whether SAS is reachable.
219 container_name: String,
220 /// Storage-key material for service-blob SAS generation
221 /// ([`presigned_get_url`](ObjectStore::presigned_get_url)).
222 /// `Some` when the credential alias resolves to a shared
223 /// account key (KEY env var or connection string); `None` for
224 /// SAS-env-var or Entra-ID paths, which return
225 /// [`ObjectStoreError::Unsupported`] for presigning.
226 sas_signing: Option<auth::SasSigningKey>,
227}
228
229impl std::fmt::Debug for AzureStore {
230 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
231 // `BlobContainerClient` is opaque (private fields, no `Debug`);
232 // surface the container URL instead so error / log lines remain
233 // useful.
234 f.debug_struct("AzureStore")
235 .field("url", &self.container.url().as_str())
236 .field("container", &self.container_name)
237 .field("sas_signing", &self.sas_signing)
238 .finish()
239 }
240}
241
242impl AzureStore {
243 /// Build an `AzureStore` from a parsed [`RemoteUrl`].
244 ///
245 /// Like the S3 backend, the [`RemoteUrl::Azure::prefix`] field is
246 /// intentionally **not** consumed here; callers compose it into keys
247 /// themselves.
248 ///
249 /// Marked `async` for symmetry with `S3Store::from_remote_url`,
250 /// which awaits the AWS provider chain. The Azure path resolves
251 /// credentials synchronously today; the signature stays `async` so
252 /// future credential providers (e.g. one that fetches an OIDC
253 /// token at construction) can plug in without breaking callers.
254 ///
255 /// # Errors
256 ///
257 /// Returns [`ObjectStoreError::Other`] if `url` is not the Azure
258 /// variant or if credential resolution fails.
259 #[allow(clippy::unused_async)]
260 pub async fn from_remote_url(url: &RemoteUrl) -> Result<Self, ObjectStoreError> {
261 let RemoteUrl::Azure {
262 endpoint,
263 account,
264 container,
265 addressing,
266 flags,
267 ..
268 } = url
269 else {
270 return Err(ObjectStoreError::Other(
271 format!("AzureStore::from_remote_url called with non-Azure URL: {url}").into(),
272 ));
273 };
274
275 let container_url = build_container_url(endpoint, account, container, *addressing);
276 let resolved = auth::resolve(account, flags)?;
277 let sas_signing = resolved.sas_signing_key.clone();
278
279 let client_options = build_client_options(&resolved)?;
280
281 let container_options = BlobContainerClientOptions {
282 client_options,
283 ..Default::default()
284 };
285
286 let container_client = BlobContainerClient::new(
287 container_url,
288 resolved.token_credential,
289 Some(container_options),
290 )
291 .map_err(other_boxed)?;
292
293 Ok(Self {
294 container: container_client,
295 container_name: container.clone(),
296 sas_signing,
297 })
298 }
299
300 /// Construct a [`BlobClient`] for an individual blob.
301 fn blob_client(&self, key: &str) -> BlobClient {
302 self.container.blob_client(key)
303 }
304
305 /// Verify the container is reachable with the configured credentials
306 /// by listing one blob (`maxresults=1`) and consuming only the first
307 /// page of results. Used by [`crate::protocol::backend::build`] to
308 /// fold credential / missing-container / authorization failures into
309 /// categorical [`crate::protocol::backend::BackendError`] variants
310 /// before the helper REPL runs its first command. Counterpart to
311 /// [`crate::object_store::s3::S3Store::probe`].
312 pub(crate) async fn probe(&self, prefix: &str) -> Result<(), ObjectStoreError> {
313 // Pass `None` for an empty prefix per the same Azurite quirk
314 // documented at the top of `list` above: a signed empty prefix
315 // returns 403 from Azurite.
316 let prefix_opt = (!prefix.is_empty()).then(|| prefix.to_owned());
317 let opts = BlobContainerClientListBlobsOptions {
318 prefix: prefix_opt,
319 maxresults: Some(1),
320 ..Default::default()
321 };
322 let mut pages = self
323 .container
324 .list_blobs(Some(opts))
325 .map_err(|e| classify(e, prefix))?
326 .into_pages();
327 // Consume only the first page: probing does not need the full
328 // listing — we only care that the request succeeded.
329 if let Some(page_result) = pages.next().await {
330 page_result.map_err(|e| classify(e, prefix))?;
331 }
332 Ok(())
333 }
334}
335
336/// Build the [`reqwest::Client`] used by [`AzureStore`]'s custom
337/// [`Transport`].
338///
339/// Bounds the connection pool's idle window, enables TCP keepalive,
340/// and sets connect / per-read timeouts so a rotated VIP cannot wedge
341/// a long-running session (see [`POOL_IDLE_TIMEOUT`] / [`TCP_KEEPALIVE`]
342/// / [`CONNECT_TIMEOUT`] / [`READ_TIMEOUT`] for rationale). Returns
343/// [`ObjectStoreError::Other`] if the TLS / DNS resolver layer fails
344/// to initialise, which the SDK would otherwise surface as a cryptic
345/// per-request error.
346pub(crate) fn build_http_client() -> Result<Arc<reqwest::Client>, ObjectStoreError> {
347 reqwest::Client::builder()
348 .pool_idle_timeout(POOL_IDLE_TIMEOUT)
349 .tcp_keepalive(TCP_KEEPALIVE)
350 .connect_timeout(CONNECT_TIMEOUT)
351 .read_timeout(READ_TIMEOUT)
352 .build()
353 .map(Arc::new)
354 .map_err(other_boxed)
355}
356
357/// Build the [`ClientOptions`] [`AzureStore`] hands to the SDK.
358///
359/// Installs the custom [`Transport`] (see [`build_http_client`]) and
360/// preserves the credential resolver's per-try signing policy. The
361/// helper is split out (rather than inlined into [`AzureStore::from_remote_url`])
362/// so unit tests can assert that both invariants hold without
363/// constructing a real `BlobContainerClient`.
364pub(crate) fn build_client_options(
365 resolved: &auth::ResolvedCredentials,
366) -> Result<ClientOptions, ObjectStoreError> {
367 let mut opts = ClientOptions {
368 transport: Some(Transport::new(build_http_client()?)),
369 ..Default::default()
370 };
371 if let Some(policy) = &resolved.per_try_policy {
372 opts.per_try_policies.push(Arc::clone(policy));
373 }
374 Ok(opts)
375}
376
377/// Construct the container-level URL [`BlobContainerClient::new`] expects.
378///
379/// The SDK addresses a container purely by URL, so any prefix segments
380/// carried by the parsed remote URL are dropped and the path is rebuilt
381/// from the account and container. For virtual-hosted addressing the
382/// path becomes `/<container>`; for path-style addressing (Azurite,
383/// custom endpoints) it becomes `/<account>/<container>`.
384pub(crate) fn build_container_url(
385 endpoint: &Url,
386 account: &str,
387 container: &str,
388 addressing: AzureAddressing,
389) -> Url {
390 let mut rewritten = endpoint.clone();
391 rewritten.set_query(None);
392 rewritten.set_fragment(None);
393 let path = match addressing {
394 AzureAddressing::VirtualHosted => format!("/{container}"),
395 AzureAddressing::PathStyle => format!("/{account}/{container}"),
396 };
397 rewritten.set_path(&path);
398 rewritten
399}
400
401/// Map an [`azure_core::Error`] into the trait's [`ObjectStoreError`] enum.
402///
403/// `key` is the operation's key/prefix context; it appears in the
404/// resulting [`ObjectStoreError::NotFound`] / [`ObjectStoreError::AccessDenied`] /
405/// [`ObjectStoreError::PreconditionFailed`] / [`ObjectStoreError::Conflict`] payload.
406fn classify(err: azure_core::Error, key: &str) -> ObjectStoreError {
407 if let azure_core::error::ErrorKind::HttpResponse {
408 status, error_code, ..
409 } = err.kind()
410 && let Some(mapped) =
411 classify_status_and_code(u16::from(*status), error_code.as_deref(), key)
412 {
413 return mapped;
414 }
415 if matches!(err.kind(), azure_core::error::ErrorKind::Io) {
416 return network_boxed(err);
417 }
418 other_boxed(err)
419}
420
421/// Pure status/code classifier (key context, no SDK types) so unit
422/// tests can exercise every branch without synthesising an SDK error.
423fn classify_status_and_code(
424 status: u16,
425 code: Option<&str>,
426 key: &str,
427) -> Option<ObjectStoreError> {
428 match status {
429 404 => return Some(ObjectStoreError::NotFound(key.to_owned())),
430 403 => return Some(ObjectStoreError::AccessDenied(key.to_owned())),
431 412 => return Some(ObjectStoreError::PreconditionFailed(key.to_owned())),
432 409 => return Some(ObjectStoreError::Conflict(key.to_owned())),
433 // Azure surfaces a Put Blob body over the single-PUT ceiling as
434 // HTTP 413 with code `RequestBodyTooLarge`; the status alone is
435 // sufficient (HTTP 413 is the canonical "Payload Too Large").
436 413 => {
437 return Some(ObjectStoreError::PayloadTooLarge {
438 limit_bytes: SINGLE_PUT_BLOB_LIMIT_BYTES,
439 });
440 }
441 _ => {}
442 }
443 // Defensive backstop for the (rare) case where the SDK exposes the
444 // service code without a 413 status: route on the code alone.
445 match code {
446 Some("RequestBodyTooLarge") => Some(ObjectStoreError::PayloadTooLarge {
447 limit_bytes: SINGLE_PUT_BLOB_LIMIT_BYTES,
448 }),
449 _ => None,
450 }
451}
452
453/// Convert the relevant `Get Blob Properties` headers into the trait's
454/// [`ObjectMeta`].
455///
456/// Extracted so unit tests can drive the missing-content-length and
457/// missing-last-modified guard branches without synthesising a full
458/// `BlobClientGetPropertiesResultHeaders` value.
459///
460/// A missing `Content-Length` is an error rather than silent zero: a
461/// 0-byte size is semantically meaningful (lock files are intentionally
462/// empty) and downstream `head_then_download` takes a fast path on
463/// `size == 0` that writes an empty destination file. Treating "header
464/// absent" as 0 would silently produce empty bundles instead of
465/// surfacing the malformed response.
466fn properties_to_meta(
467 key: &str,
468 content_length: Option<u64>,
469 last_modified: Option<OffsetDateTime>,
470 etag: Option<&str>,
471) -> Result<ObjectMeta, ObjectStoreError> {
472 let size = content_length.ok_or_else(|| {
473 ObjectStoreError::Other(
474 format!("get_properties on `{key}` returned no content-length").into(),
475 )
476 })?;
477 let last_modified = last_modified.ok_or_else(|| {
478 ObjectStoreError::Other(
479 format!("get_properties on `{key}` returned no last-modified").into(),
480 )
481 })?;
482 Ok(ObjectMeta {
483 key: key.to_owned(),
484 size,
485 last_modified,
486 etag: etag.map(str::to_owned),
487 })
488}
489
490/// Convert a `BlobItem`-shaped record into the trait's [`ObjectMeta`].
491///
492/// Extracted so unit tests can drive the missing-field guards without
493/// synthesising a full `ListBlobsResponse`.
494fn item_to_meta(
495 name: Option<&str>,
496 content_length: Option<u64>,
497 last_modified: Option<OffsetDateTime>,
498 etag: Option<&str>,
499) -> Result<ObjectMeta, ObjectStoreError> {
500 let key = name
501 .ok_or_else(|| ObjectStoreError::Other("list_blobs returned a blob without a name".into()))?
502 .to_owned();
503 let size = content_length.unwrap_or(0);
504 let last_modified = last_modified.ok_or_else(|| {
505 ObjectStoreError::Other(
506 format!("list_blobs returned blob `{key}` without last_modified").into(),
507 )
508 })?;
509 Ok(ObjectMeta {
510 key,
511 size,
512 last_modified,
513 etag: etag.map(str::to_owned),
514 })
515}
516
517#[async_trait::async_trait]
518impl ObjectStore for AzureStore {
519 async fn list(&self, prefix: &str) -> Result<Vec<ObjectMeta>, ObjectStoreError> {
520 // Pass `None` for an empty prefix: Azure list_blobs URL-encodes
521 // `prefix=` and Azurite signs an empty value differently than
522 // an absent one (treats it as a tampered query and returns
523 // 403). Skipping the parameter is the wire-equivalent of "no
524 // prefix filter" anyway.
525 let prefix_opt = (!prefix.is_empty()).then(|| prefix.to_owned());
526 let opts = BlobContainerClientListBlobsOptions {
527 prefix: prefix_opt,
528 ..Default::default()
529 };
530 let mut pages = self
531 .container
532 .list_blobs(Some(opts))
533 .map_err(|e| classify(e, prefix))?
534 .into_pages();
535
536 let mut out = Vec::new();
537 while let Some(page_result) = pages.next().await {
538 let response = page_result.map_err(|e| classify(e, prefix))?;
539 let body = response
540 .into_body()
541 .xml::<azure_storage_blob::models::ListBlobsResponse>()
542 .map_err(|e| classify(e, prefix))?;
543 for item in body.blob_items {
544 let props = item.properties.unwrap_or_default();
545 let meta = item_to_meta(
546 item.name.as_deref(),
547 props.content_length,
548 props.last_modified,
549 // Listing omits ETag for parity with S3 (avoid
550 // inflating per-object metadata for callers that
551 // only need a key/size enumeration).
552 None,
553 )?;
554 out.push(meta);
555 }
556 }
557 Ok(out)
558 }
559
560 async fn get_to_file(
561 &self,
562 key: &str,
563 dest: &Path,
564 opts: GetOpts,
565 ) -> Result<(), ObjectStoreError> {
566 let parent = dest.parent().ok_or_else(|| {
567 ObjectStoreError::Other(
568 format!("destination `{}` has no parent directory", dest.display()).into(),
569 )
570 })?;
571
572 // Mirror S3: try once, retry once on 412 (the head→GET race).
573 // After the second attempt any error — including a repeated
574 // 412 — propagates.
575 let progress = opts.progress.as_ref();
576 match self.head_then_download(key, dest, parent, progress).await {
577 Err(ObjectStoreError::PreconditionFailed(_)) => {
578 tracing::warn!(key, "blob changed between head and GET; retrying");
579 self.head_then_download(key, dest, parent, progress).await
580 }
581 other => other,
582 }
583 }
584
585 async fn get_bytes(&self, key: &str) -> Result<Bytes, ObjectStoreError> {
586 let blob = self.blob_client(key);
587 let result = blob.download(None).await.map_err(|e| classify(e, key))?;
588 let bytes = result.body.collect().await.map_err(network_boxed)?;
589 Ok(bytes)
590 }
591
592 /// Issue a Get Blob with an `HttpRange` covering `[start, end)`.
593 /// HTTP 416 maps to [`ObjectStoreError::RangeNotSatisfiable`] with
594 /// the original `Range<u64>` so the wire-line names what the
595 /// caller asked for. All other failures route through [`classify`].
596 ///
597 /// Azure silently truncates a ranged GET to EOF when the requested
598 /// range overruns the blob — `start < body.len() <= end` returns
599 /// `start..body.len()` bytes with HTTP 206 and no error. The
600 /// post-flight length check via [`super::verify_range_response_length`]
601 /// elevates that mismatch to [`ObjectStoreError::RangeNotSatisfiable`]
602 /// so callers (notably the packchain reader) cannot mistake a
603 /// truncated slice for the full requested range.
604 async fn get_bytes_range(
605 &self,
606 key: &str,
607 range: std::ops::Range<u64>,
608 ) -> Result<Bytes, ObjectStoreError> {
609 if let Some(empty) = super::precheck_range(key, &range)? {
610 return Ok(empty);
611 }
612 // `precheck_range` has already rejected `start > end`, so the
613 // `HttpRange` length below cannot underflow.
614 let opts = BlobClientDownloadOptions {
615 range: Some(HttpRange::from(range.clone())),
616 ..Default::default()
617 };
618 let blob = self.blob_client(key);
619 let result = match blob.download(Some(opts)).await {
620 Ok(result) => result,
621 Err(err) => {
622 if let azure_core::error::ErrorKind::HttpResponse { status, .. } = err.kind()
623 && u16::from(*status) == 416
624 {
625 return Err(ObjectStoreError::RangeNotSatisfiable {
626 key: key.to_owned(),
627 requested: range,
628 });
629 }
630 return Err(classify(err, key));
631 }
632 };
633 let bytes = result.body.collect().await.map_err(network_boxed)?;
634 super::verify_range_response_length(key, &range, bytes)
635 }
636
637 async fn put_bytes(
638 &self,
639 key: &str,
640 body: Bytes,
641 opts: PutOpts,
642 ) -> Result<(), ObjectStoreError> {
643 // Same threshold as the S3 backend: above
644 // [`MULTIPART_PUT_THRESHOLD`] use explicit `stage_block` +
645 // `commit_block_list` so each block has its own retry budget,
646 // predictable concurrency, and per-block progress events. Below
647 // the threshold keep the single `Put Blob` round trip. Issue #53.
648 let size = body.len() as u64;
649 if should_use_multipart(size) {
650 return self.multipart_put_bytes(key, body, size, opts).await;
651 }
652 let progress = opts.progress.clone();
653 let blob = self.blob_client(key);
654 let upload_opts = upload_options_from(opts);
655 blob.upload(bytes_to_request_content(body), Some(upload_opts))
656 .await
657 .map_err(|e| classify(e, key))?;
658 if let Some(sink) = progress
659 && size > 0
660 {
661 sink.report(size);
662 }
663 Ok(())
664 }
665
666 /// Stream a local file to `key` without buffering its full body.
667 ///
668 /// Above [`super::multipart::MULTIPART_PUT_THRESHOLD`] this routes through explicit
669 /// `stage_block` + `commit_block_list`, paralleling the S3 backend
670 /// (issue #53). Below the threshold the single `Put Blob` path
671 /// preserves the one-round-trip cost for small bundles and lock
672 /// files.
673 ///
674 /// On the multipart path each task opens its own
675 /// `tokio::fs::File`, seeks to its part offset, reads the part
676 /// into a `Bytes`, then calls `BlockBlobClient::stage_block`. With
677 /// `MULTIPART_PUT_MAX_CONCURRENCY = 8` and
678 /// `MULTIPART_PUT_PART_SIZE = 16 MiB`, peak memory is bounded at
679 /// 128 MiB regardless of file size.
680 ///
681 /// On the single-PUT path we wrap `tokio::fs::File` in
682 /// [`FileStream`] so the body is delivered as
683 /// `Body::SeekableStream`. The per-try signing policy reads
684 /// `request.body().len()`, which `SeekableStream` reports faithfully
685 /// via `len()`.
686 async fn put_path(&self, key: &str, src: &Path, opts: PutOpts) -> Result<(), ObjectStoreError> {
687 // Open the file once and read size from the open handle. This
688 // closes the metadata/upload race that would let a concurrent
689 // truncate or rename produce a body whose length disagrees
690 // with the size we used for multipart planning.
691 let file = tokio::fs::File::open(src).await.map_err(other_boxed)?;
692 let body_len = file.metadata().await.map_err(other_boxed)?.len();
693 if should_use_multipart(body_len) {
694 return self.multipart_put_path(key, file, body_len, opts).await;
695 }
696 // Below the threshold: single `Put Blob`. Wrap our already-
697 // open handle in `FileStream`; the SDK does not re-open by
698 // path (which would re-introduce the race).
699 let stream = FileStream::builder(file)
700 .build()
701 .await
702 .map_err(other_boxed)?;
703 let body: azure_core::http::Body = stream.into();
704
705 let blob = self.blob_client(key);
706 let progress = opts.progress.clone();
707 let upload_opts = upload_options_from(opts);
708 blob.upload(body.into(), Some(upload_opts))
709 .await
710 .map_err(|e| classify(e, key))?;
711 if let Some(sink) = progress
712 && body_len > 0
713 {
714 sink.report(body_len);
715 }
716 Ok(())
717 }
718
719 async fn put_if_absent(&self, key: &str, body: Bytes) -> Result<bool, ObjectStoreError> {
720 let blob = self.blob_client(key);
721 let upload_opts = BlockBlobClientUploadOptions::default().if_not_exists();
722 let resp = blob
723 .upload(bytes_to_request_content(body), Some(upload_opts))
724 .await;
725 match resp.map_err(|e| classify(e, key)) {
726 Ok(_) => Ok(true),
727 Err(ObjectStoreError::PreconditionFailed(_) | ObjectStoreError::Conflict(_)) => {
728 Ok(false)
729 }
730 Err(other) => Err(other),
731 }
732 }
733
734 async fn head(&self, key: &str) -> Result<ObjectMeta, ObjectStoreError> {
735 let blob = self.blob_client(key);
736 let resp = blob
737 .get_properties(None::<BlobClientGetPropertiesOptions<'_>>)
738 .await
739 .map_err(|e| classify(e, key))?;
740 let headers = resp.headers();
741 properties_to_meta(
742 key,
743 header_u64(headers, &HeaderName::from_static("content-length")),
744 header_http_date(headers, &HeaderName::from_static("last-modified")),
745 headers.get_optional_str(&HeaderName::from_static("etag")),
746 )
747 }
748
749 async fn copy(&self, src: &str, dst: &str) -> Result<(), ObjectStoreError> {
750 // Server-side copy via `Put Blob From URL` requires a SAS-tokened
751 // source URL or `x-ms-copy-source-authorization`, neither of
752 // which integrates with our credential model in a clean way
753 // for the SDK 1.0 surface. Stream `src` to a temp file via
754 // `get_to_file` (chunked download, no body buffer), then
755 // `put_path` it back to `dst` (block-uploaded for large bodies
756 // via `multipart_put_path`). Peak in-flight bytes are bounded
757 // by `MULTIPART_PUT_MAX_CONCURRENCY` × `MULTIPART_PUT_PART_SIZE`
758 // regardless of blob size — necessary because `manage doctor`'s
759 // duplicate-bundle quarantine path uses `copy()` and bundles
760 // can be multi-GiB.
761 let temp = NamedTempFile::new().map_err(other_boxed)?;
762 // `get_to_file` propagates `NotFound(src)` if the source is
763 // absent — exactly the trait contract for `copy`.
764 self.get_to_file(src, temp.path(), GetOpts::default())
765 .await?;
766 // A NotFound on the upload is destination-side — re-shape it
767 // so callers don't mistake it for "src absent".
768 match self.put_path(dst, temp.path(), PutOpts::default()).await {
769 Ok(()) => Ok(()),
770 Err(ObjectStoreError::NotFound(_)) => Err(ObjectStoreError::Other(
771 format!("copy `{src}` → `{dst}`: upload returned NotFound").into(),
772 )),
773 Err(other) => Err(other),
774 }
775 }
776
777 async fn delete(&self, key: &str) -> Result<(), ObjectStoreError> {
778 let blob = self.blob_client(key);
779 blob.delete(None::<BlobClientDeleteOptions<'_>>)
780 .await
781 .map_err(|e| classify(e, key))?;
782 Ok(())
783 }
784
785 /// Build a service-blob SAS URL for `key` valid for `ttl`.
786 /// Used by the `bundle-uri` capability (issue #76) to advertise
787 /// time-limited download URLs against private containers.
788 ///
789 /// Only the shared-key / connection-string credential paths can
790 /// produce a SAS — the SAS env-var path has no key to re-sign
791 /// with, and the Entra-ID `TokenCredential` path requires
792 /// user-delegation SAS (out of scope per the issue). Both
793 /// fall through to [`ObjectStoreError::Unsupported`].
794 ///
795 /// # Errors
796 ///
797 /// - [`ObjectStoreError::Unsupported`] when the credential is
798 /// not a shared key.
799 /// - [`ObjectStoreError::Other`] when SAS construction fails
800 /// (HMAC init / base64 decode / time overflow).
801 async fn presigned_get_url(
802 &self,
803 key: &str,
804 ttl: std::time::Duration,
805 ) -> Result<String, ObjectStoreError> {
806 let signing = self.sas_signing.as_ref().ok_or_else(|| {
807 ObjectStoreError::Unsupported(
808 "Azure presigned URLs require a shared account key (KEY env var or \
809 connection string); SAS-env-var and Entra-ID credentials cannot \
810 derive per-blob SAS"
811 .to_owned(),
812 )
813 })?;
814 // The SDK's `BlobClient::url()` returns the fully-qualified
815 // blob URL including the container path segment. Reuse it
816 // rather than re-deriving the URL shape per addressing
817 // mode here.
818 let blob = self.blob_client(key);
819 let base = blob.url();
820 sas::build_blob_sas_url(base, &self.container_name, key, signing, ttl)
821 }
822}
823
824impl AzureStore {
825 /// One head→tempfile→download→persist round trip.
826 ///
827 /// Factored out so [`get_to_file`](ObjectStore::get_to_file) can
828 /// invoke it twice: once normally, once more on a 412 retry.
829 async fn head_then_download(
830 &self,
831 key: &str,
832 dest: &Path,
833 parent: &Path,
834 progress: Option<&ProgressSink>,
835 ) -> Result<(), ObjectStoreError> {
836 let meta = self.head(key).await?;
837 let temp = NamedTempFile::new_in(parent).map_err(other_boxed)?;
838 if meta.size == 0 {
839 // Skip the GET entirely for zero-byte blobs (lock files):
840 // `download_streaming` would issue a plain GET for an empty
841 // body — correct but a wasted round trip.
842 return persist_temp(temp, dest);
843 }
844 self.download_streaming(key, temp.path(), meta.etag.as_deref(), progress)
845 .await?;
846 persist_temp(temp, dest)
847 }
848
849 /// Stream a blob body to `temp_path` with optional `If-Match`
850 /// guarding against mid-download mutation. When `progress` is
851 /// `Some`, fires once per SDK body chunk read off the wire.
852 async fn download_streaming(
853 &self,
854 key: &str,
855 temp_path: &Path,
856 etag: Option<&str>,
857 progress: Option<&ProgressSink>,
858 ) -> Result<(), ObjectStoreError> {
859 let blob = self.blob_client(key);
860 let mut opts = BlobClientDownloadOptions::default();
861 if let Some(etag) = etag {
862 opts.if_match = Some(etag.into());
863 }
864 let mut result = blob
865 .download(Some(opts))
866 .await
867 .map_err(|e| classify(e, key))?;
868
869 let mut file = tokio::fs::OpenOptions::new()
870 .write(true)
871 .truncate(true)
872 .open(temp_path)
873 .await
874 .map_err(other_boxed)?;
875
876 while let Some(chunk) = result.body.next().await {
877 let bytes = chunk.map_err(network_boxed)?;
878 let chunk_len = bytes.len() as u64;
879 file.write_all(&bytes).await.map_err(other_boxed)?;
880 if let Some(sink) = progress
881 && chunk_len > 0
882 {
883 sink.report(chunk_len);
884 }
885 }
886 file.flush().await.map_err(other_boxed)?;
887 Ok(())
888 }
889
890 /// Drive a multipart upload from a fully-buffered `Bytes` body.
891 ///
892 /// `Bytes::slice` is zero-copy — every block borrows into the same
893 /// underlying allocation, so peak memory equals the caller's body
894 /// rather than `body × blocks`.
895 async fn multipart_put_bytes(
896 &self,
897 key: &str,
898 body: Bytes,
899 size: u64,
900 opts: PutOpts,
901 ) -> Result<(), ObjectStoreError> {
902 let parts = plan_upload_parts(size, MULTIPART_PUT_PART_SIZE, AZURE_MAX_BLOCKS);
903 let progress = opts.progress.clone();
904 let staged = self
905 .stage_blocks_with_bodies(key, &parts, progress, |part| slice_bytes_part(&body, part))
906 .await?;
907 let blob = self.blob_client(key).block_blob_client();
908 commit_block_list(&blob, key, staged, opts).await
909 }
910
911 /// Drive a multipart upload by streaming a local file block-by-block.
912 ///
913 /// All tasks share one `Arc<std::fs::File>`; per-task
914 /// `read_file_part` uses `pread` so reads are concurrent without
915 /// offset contention. Sharing one open file description closes
916 /// the metadata/upload race. With `MULTIPART_PUT_MAX_CONCURRENCY
917 /// = 8` and `MULTIPART_PUT_PART_SIZE = 16 MiB`, peak memory is
918 /// bounded at 128 MiB regardless of file size.
919 async fn multipart_put_path(
920 &self,
921 key: &str,
922 file: tokio::fs::File,
923 size: u64,
924 opts: PutOpts,
925 ) -> Result<(), ObjectStoreError> {
926 let parts = plan_upload_parts(size, MULTIPART_PUT_PART_SIZE, AZURE_MAX_BLOCKS);
927 let progress = opts.progress.clone();
928 let file: Arc<std::fs::File> = Arc::new(file.into_std().await);
929 let staged = self
930 .stage_blocks_from_file(key, file, &parts, progress)
931 .await?;
932 let blob = self.blob_client(key).block_blob_client();
933 commit_block_list(&blob, key, staged, opts).await
934 }
935
936 /// Spawn parallel `stage_block` tasks with bodies sourced from a
937 /// closure (used by `multipart_put_bytes`).
938 ///
939 /// Returns the per-block IDs in part order so the caller can build
940 /// a `BlockLookupList` for `commit_block_list`. On error,
941 /// already-staged blocks are simply not committed and Azure
942 /// auto-expires them after seven days; there is no client-side
943 /// abort call.
944 ///
945 /// `BlockBlobClient` does not implement `Clone`, so each spawned
946 /// task constructs its own via `self.blob_client(...)
947 /// .block_blob_client()`. The container's `blob_client(&self, ..)`
948 /// returns an owned `BlobClient` already (cheap-clone of internal
949 /// `Arc` state), so this stays allocation-light.
950 async fn stage_blocks_with_bodies<F>(
951 &self,
952 key: &str,
953 parts: &[UploadPart],
954 progress: Option<ProgressSink>,
955 make_body: F,
956 ) -> Result<Vec<Vec<u8>>, ObjectStoreError>
957 where
958 F: Fn(UploadPart) -> Result<Bytes, ObjectStoreError>,
959 {
960 let semaphore = Arc::new(Semaphore::new(MULTIPART_PUT_MAX_CONCURRENCY));
961 let mut tasks: JoinSet<Result<(usize, Vec<u8>), ObjectStoreError>> = JoinSet::new();
962 for (idx, part) in parts.iter().enumerate() {
963 let part = *part;
964 let part_index = idx;
965 let block_id = block_id_for(idx);
966 let body = make_body(part)?;
967 let blob = self.blob_client(key).block_blob_client();
968 let key = key.to_owned();
969 let semaphore = Arc::clone(&semaphore);
970 let progress = progress.clone();
971 tasks.spawn(async move {
972 let _permit = semaphore.acquire_owned().await.map_err(other_boxed)?;
973 blob.stage_block(&block_id, part.length, bytes_to_request_content(body), None)
974 .await
975 .map_err(|e| classify(e, &key))?;
976 if let Some(sink) = &progress {
977 sink.report(part.length);
978 }
979 Ok((part_index, block_id))
980 });
981 }
982 join_staged_blocks(tasks, parts.len()).await
983 }
984
985 /// Spawn parallel `stage_block` tasks that each read their
986 /// block from the shared `Arc<std::fs::File>` via `pread`. The
987 /// shared open file description gives every task a stable view
988 /// of the same inode (used by `multipart_put_path`).
989 async fn stage_blocks_from_file(
990 &self,
991 key: &str,
992 file: Arc<std::fs::File>,
993 parts: &[UploadPart],
994 progress: Option<ProgressSink>,
995 ) -> Result<Vec<Vec<u8>>, ObjectStoreError> {
996 let semaphore = Arc::new(Semaphore::new(MULTIPART_PUT_MAX_CONCURRENCY));
997 let mut tasks: JoinSet<Result<(usize, Vec<u8>), ObjectStoreError>> = JoinSet::new();
998 for (idx, part) in parts.iter().enumerate() {
999 let part = *part;
1000 let part_index = idx;
1001 let block_id = block_id_for(idx);
1002 let blob = self.blob_client(key).block_blob_client();
1003 let key = key.to_owned();
1004 let task_file = Arc::clone(&file);
1005 let semaphore = Arc::clone(&semaphore);
1006 let progress = progress.clone();
1007 tasks.spawn(async move {
1008 let _permit = semaphore.acquire_owned().await.map_err(other_boxed)?;
1009 let body = read_file_part(task_file, part).await?;
1010 blob.stage_block(&block_id, part.length, bytes_to_request_content(body), None)
1011 .await
1012 .map_err(|e| classify(e, &key))?;
1013 if let Some(sink) = &progress {
1014 sink.report(part.length);
1015 }
1016 Ok((part_index, block_id))
1017 });
1018 }
1019 join_staged_blocks(tasks, parts.len()).await
1020 }
1021}
1022
1023/// Build a deterministic Azure block ID for the `idx`-th part
1024/// (zero-indexed).
1025///
1026/// Azure requires that all block IDs in a single
1027/// `commit_block_list` request share a length pre-base64. 32 bytes
1028/// of zero-padded ASCII digits accommodates up to 10^32 parts —
1029/// vastly above [`AZURE_MAX_BLOCKS`] = 50 000.
1030fn block_id_for(idx: usize) -> Vec<u8> {
1031 format!("{:032}", idx + 1).into_bytes()
1032}
1033
1034/// Drain a `JoinSet` of `stage_block` tasks into a Vec of block IDs
1035/// indexed by part order. Short-circuits on the first error.
1036async fn join_staged_blocks(
1037 mut tasks: JoinSet<Result<(usize, Vec<u8>), ObjectStoreError>>,
1038 expected: usize,
1039) -> Result<Vec<Vec<u8>>, ObjectStoreError> {
1040 let mut staged: Vec<Option<Vec<u8>>> = (0..expected).map(|_| None).collect();
1041 while let Some(joined) = tasks.join_next().await {
1042 let (idx, block_id) = joined.map_err(other_boxed)??;
1043 staged[idx] = Some(block_id);
1044 }
1045 staged
1046 .into_iter()
1047 .enumerate()
1048 .map(|(idx, slot)| {
1049 slot.ok_or_else(|| {
1050 ObjectStoreError::Other(
1051 format!("internal: stage_block task for part {idx} did not return").into(),
1052 )
1053 })
1054 })
1055 .collect()
1056}
1057
1058/// Commit the staged blocks in order, applying any
1059/// `content_disposition` / `user_metadata` from the original `PutOpts`.
1060///
1061/// Azure has no `AbortMultipartUpload` equivalent: if commit fails
1062/// the staged blocks remain on the storage account and expire
1063/// automatically (default seven days). Surface the commit error
1064/// directly — the caller's error handling already understands the
1065/// "operation did not succeed" outcome.
1066async fn commit_block_list(
1067 blob: &BlockBlobClient,
1068 key: &str,
1069 block_ids: Vec<Vec<u8>>,
1070 opts: PutOpts,
1071) -> Result<(), ObjectStoreError> {
1072 let block_list = BlockLookupList {
1073 latest: Some(block_ids),
1074 ..Default::default()
1075 };
1076 let body: RequestContent<_, _> = block_list.try_into().map_err(other_boxed)?;
1077 let (cd, metadata) = put_opts_blob_fields(opts);
1078 let commit_opts = BlockBlobClientCommitBlockListOptions {
1079 blob_content_disposition: cd,
1080 metadata,
1081 ..Default::default()
1082 };
1083 blob.commit_block_list(body, Some(commit_opts))
1084 .await
1085 .map_err(|e| classify(e, key))?;
1086 Ok(())
1087}
1088
1089/// Wrap `Bytes` in a `RequestContent` without copying the buffer.
1090///
1091/// `RequestContent` has an inherent `from(Vec<u8>)` constructor that
1092/// shadows the generic `From<Bytes>` trait impl, so a bare
1093/// `RequestContent::from(body)` resolves to the `Vec<u8>` overload and
1094/// re-allocates. Going through `Into` instead picks up the trait impl
1095/// and keeps the `Bytes` payload zero-copy. The return type is left
1096/// generic so the call site (which pins `Bytes` + `NoFormat` via the
1097/// `BlobClient::upload` signature) drives type inference.
1098fn bytes_to_request_content<F>(body: Bytes) -> RequestContent<Bytes, F>
1099where
1100 Bytes: Into<RequestContent<Bytes, F>>,
1101{
1102 body.into()
1103}
1104
1105/// Pull the blob-shaped `content_disposition` and `metadata` fields
1106/// out of [`PutOpts`].
1107///
1108/// Both `BlockBlobClientUploadOptions` (single `Put Blob`) and
1109/// `BlockBlobClientCommitBlockListOptions` (multipart commit) carry
1110/// the same two fields by the same names. Centralising the
1111/// conversion here keeps a single source of truth for "how a
1112/// `PutOpts` becomes Azure blob metadata."
1113fn put_opts_blob_fields(
1114 opts: PutOpts,
1115) -> (
1116 Option<String>,
1117 Option<std::collections::HashMap<String, String>>,
1118) {
1119 let metadata = (!opts.user_metadata.is_empty()).then(|| {
1120 opts.user_metadata
1121 .into_iter()
1122 .collect::<std::collections::HashMap<_, _>>()
1123 });
1124 (opts.content_disposition, metadata)
1125}
1126
1127/// Build a [`BlockBlobClientUploadOptions`] from the trait's [`PutOpts`].
1128fn upload_options_from(opts: PutOpts) -> BlockBlobClientUploadOptions<'static> {
1129 let (cd, metadata) = put_opts_blob_fields(opts);
1130 BlockBlobClientUploadOptions {
1131 blob_content_disposition: cd,
1132 metadata,
1133 ..Default::default()
1134 }
1135}
1136
1137fn header_u64(headers: &Headers, name: &HeaderName) -> Option<u64> {
1138 headers.get_optional_str(name).and_then(|s| s.parse().ok())
1139}
1140
1141fn header_http_date(headers: &Headers, name: &HeaderName) -> Option<OffsetDateTime> {
1142 let raw = headers.get_optional_str(name)?;
1143 OffsetDateTime::parse(raw, &time::format_description::well_known::Rfc2822).ok()
1144}
1145
1146#[cfg(test)]
1147mod tests {
1148 use super::*;
1149 use crate::url::{AzureAddressing, RemoteFlags};
1150
1151 fn parse_endpoint(s: &str) -> Url {
1152 Url::parse(s).expect("test endpoint URL parses")
1153 }
1154
1155 fn s3_url() -> RemoteUrl {
1156 RemoteUrl::S3 {
1157 endpoint: parse_endpoint("https://my-bucket.s3.us-west-2.amazonaws.com/"),
1158 bucket: "my-bucket".to_owned(),
1159 prefix: None,
1160 addressing: crate::url::S3Addressing::VirtualHosted,
1161 flags: RemoteFlags::default(),
1162 }
1163 }
1164
1165 // --- build_container_url ------------------------------------------
1166
1167 #[test]
1168 fn build_container_url_virtual_hosted_strips_prefix() {
1169 let url = parse_endpoint("https://acct.blob.core.windows.net/my-container/some/prefix");
1170 let out = build_container_url(&url, "acct", "my-container", AzureAddressing::VirtualHosted);
1171 assert_eq!(
1172 out.as_str(),
1173 "https://acct.blob.core.windows.net/my-container"
1174 );
1175 }
1176
1177 #[test]
1178 fn build_container_url_path_style_keeps_account() {
1179 let url = parse_endpoint("http://127.0.0.1:10000/devstoreaccount1/my-container/repo");
1180 let out = build_container_url(
1181 &url,
1182 "devstoreaccount1",
1183 "my-container",
1184 AzureAddressing::PathStyle,
1185 );
1186 assert_eq!(
1187 out.as_str(),
1188 "http://127.0.0.1:10000/devstoreaccount1/my-container"
1189 );
1190 }
1191
1192 #[test]
1193 fn build_container_url_strips_query_and_fragment() {
1194 let url = parse_endpoint("https://acct.blob.core.windows.net/c/r?credential=foo#frag");
1195 let out = build_container_url(&url, "acct", "c", AzureAddressing::VirtualHosted);
1196 assert_eq!(out.as_str(), "https://acct.blob.core.windows.net/c");
1197 }
1198
1199 // --- classify_status_and_code -------------------------------------
1200
1201 #[test]
1202 fn classify_404_is_not_found() {
1203 assert!(matches!(
1204 classify_status_and_code(404, None, "k"),
1205 Some(ObjectStoreError::NotFound(s)) if s == "k"
1206 ));
1207 }
1208
1209 #[test]
1210 fn classify_403_is_access_denied() {
1211 assert!(matches!(
1212 classify_status_and_code(403, None, "k"),
1213 Some(ObjectStoreError::AccessDenied(s)) if s == "k"
1214 ));
1215 }
1216
1217 #[test]
1218 fn classify_412_is_precondition_failed() {
1219 assert!(matches!(
1220 classify_status_and_code(412, None, "k"),
1221 Some(ObjectStoreError::PreconditionFailed(s)) if s == "k"
1222 ));
1223 }
1224
1225 #[test]
1226 fn classify_409_is_conflict() {
1227 // 409 covers Azure's `BlobAlreadyExists` (the put-if-absent
1228 // contention path). Without this branch, `put_if_absent` would
1229 // surface contention as a hard error instead of `Ok(false)`.
1230 assert!(matches!(
1231 classify_status_and_code(409, None, "k"),
1232 Some(ObjectStoreError::Conflict(s)) if s == "k"
1233 ));
1234 }
1235
1236 #[test]
1237 fn classify_413_is_payload_too_large() {
1238 // Pass `code=None` so the assertion isolates the 413-status
1239 // branch; passing a recognised code would still pass even if
1240 // the status arm regressed (the code arm would catch it). The
1241 // canonical "Payload Too Large" status alone suffices.
1242 assert!(matches!(
1243 classify_status_and_code(413, None, "k"),
1244 Some(ObjectStoreError::PayloadTooLarge { limit_bytes })
1245 if limit_bytes == SINGLE_PUT_BLOB_LIMIT_BYTES
1246 ));
1247 }
1248
1249 #[test]
1250 fn classify_request_body_too_large_code_is_payload_too_large() {
1251 // Defensive backstop: if the SDK delivers the service code on a
1252 // non-413 status (e.g. 400), the code branch still catches it.
1253 assert!(matches!(
1254 classify_status_and_code(400, Some("RequestBodyTooLarge"), "k"),
1255 Some(ObjectStoreError::PayloadTooLarge { limit_bytes })
1256 if limit_bytes == SINGLE_PUT_BLOB_LIMIT_BYTES
1257 ));
1258 }
1259
1260 #[test]
1261 fn classify_unrecognised_status_returns_none() {
1262 assert!(classify_status_and_code(500, None, "k").is_none());
1263 assert!(classify_status_and_code(429, None, "k").is_none());
1264 assert!(classify_status_and_code(500, Some("InternalError"), "k").is_none());
1265 }
1266
1267 // --- properties_to_meta ------------------------------------------
1268
1269 #[test]
1270 fn properties_to_meta_round_trips_well_formed_response() {
1271 let now = OffsetDateTime::from_unix_timestamp(1_700_000_000).unwrap();
1272 let meta = properties_to_meta("k", Some(42), Some(now), Some("\"abc\""))
1273 .expect("conversion succeeds");
1274 assert_eq!(meta.key, "k");
1275 assert_eq!(meta.size, 42);
1276 assert_eq!(meta.last_modified.unix_timestamp(), 1_700_000_000);
1277 assert_eq!(meta.etag.as_deref(), Some("\"abc\""));
1278 }
1279
1280 #[test]
1281 fn properties_to_meta_preserves_legitimate_zero_size() {
1282 // Zero-byte lock files are legitimate; a present
1283 // `Content-Length: 0` header (`Some(0)`) must round-trip as
1284 // `size == 0`, distinct from the missing-header error.
1285 let now = OffsetDateTime::from_unix_timestamp(1_700_000_000).unwrap();
1286 let meta =
1287 properties_to_meta("LOCK", Some(0), Some(now), None).expect("conversion succeeds");
1288 assert_eq!(meta.size, 0);
1289 }
1290
1291 #[test]
1292 fn properties_to_meta_rejects_missing_content_length() {
1293 let now = OffsetDateTime::from_unix_timestamp(1_700_000_000).unwrap();
1294 let err = properties_to_meta("k", None, Some(now), None)
1295 .expect_err("missing content-length must error");
1296 match err {
1297 ObjectStoreError::Other(inner) => {
1298 let msg = inner.to_string();
1299 assert!(msg.contains("no content-length"), "names failure: {msg}");
1300 assert!(msg.contains("`k`"), "includes the key for context: {msg}");
1301 }
1302 other => {
1303 panic!("expected ObjectStoreError::Other for missing content-length, got {other:?}")
1304 }
1305 }
1306 }
1307
1308 #[test]
1309 fn properties_to_meta_rejects_missing_last_modified() {
1310 let err = properties_to_meta("k", Some(0), None, None)
1311 .expect_err("missing last_modified must error");
1312 match err {
1313 ObjectStoreError::Other(inner) => {
1314 let msg = inner.to_string();
1315 assert!(msg.contains("no last-modified"), "names failure: {msg}");
1316 assert!(msg.contains("`k`"), "includes the key for context: {msg}");
1317 }
1318 other => {
1319 panic!("expected ObjectStoreError::Other for missing last_modified, got {other:?}")
1320 }
1321 }
1322 }
1323
1324 // --- item_to_meta -------------------------------------------------
1325
1326 #[test]
1327 fn item_to_meta_round_trips_well_formed_item() {
1328 let now = OffsetDateTime::from_unix_timestamp(1_700_000_000).unwrap();
1329 let meta = item_to_meta(Some("k"), Some(42), Some(now), Some("\"abc\"")).unwrap();
1330 assert_eq!(meta.key, "k");
1331 assert_eq!(meta.size, 42);
1332 assert_eq!(meta.last_modified.unix_timestamp(), 1_700_000_000);
1333 assert_eq!(meta.etag.as_deref(), Some("\"abc\""));
1334 }
1335
1336 #[test]
1337 fn item_to_meta_rejects_missing_name() {
1338 let now = OffsetDateTime::from_unix_timestamp(1_700_000_000).unwrap();
1339 let err = item_to_meta(None, Some(0), Some(now), None).unwrap_err();
1340 match err {
1341 ObjectStoreError::Other(inner) => {
1342 assert!(
1343 inner.to_string().contains("without a name"),
1344 "names failure: {inner}"
1345 );
1346 }
1347 other => panic!("expected ObjectStoreError::Other, got {other:?}"),
1348 }
1349 }
1350
1351 #[test]
1352 fn item_to_meta_rejects_missing_last_modified() {
1353 let err = item_to_meta(Some("k"), Some(0), None, None).unwrap_err();
1354 match err {
1355 ObjectStoreError::Other(inner) => {
1356 let msg = inner.to_string();
1357 assert!(
1358 msg.contains("without last_modified"),
1359 "names failure: {msg}"
1360 );
1361 assert!(msg.contains("`k`"), "includes the key: {msg}");
1362 }
1363 other => panic!("expected ObjectStoreError::Other, got {other:?}"),
1364 }
1365 }
1366
1367 #[test]
1368 fn item_to_meta_treats_missing_size_as_zero() {
1369 // The Azure SDK types content_length as Option<u64>; missing
1370 // values default to 0 (rather than `None` propagating through
1371 // every caller's arithmetic).
1372 let now = OffsetDateTime::from_unix_timestamp(1_700_000_000).unwrap();
1373 let meta = item_to_meta(Some("k"), None, Some(now), None).unwrap();
1374 assert_eq!(meta.size, 0);
1375 }
1376
1377 // --- upload_options_from ------------------------------------------
1378
1379 #[test]
1380 fn upload_options_from_default_is_empty() {
1381 let out = upload_options_from(PutOpts::default());
1382 assert!(out.blob_content_disposition.is_none());
1383 assert!(out.metadata.is_none());
1384 }
1385
1386 #[test]
1387 fn upload_options_from_carries_content_disposition() {
1388 let opts = PutOpts {
1389 content_disposition: Some("attachment; filename=x".into()),
1390 user_metadata: Vec::new(),
1391 progress: None,
1392 };
1393 let out = upload_options_from(opts);
1394 let cd: String = out
1395 .blob_content_disposition
1396 .expect("content_disposition should be set");
1397 assert!(cd.contains("attachment"));
1398 }
1399
1400 #[test]
1401 fn upload_options_from_collects_metadata() {
1402 let opts = PutOpts {
1403 content_disposition: None,
1404 user_metadata: vec![("x-foo".into(), "1".into()), ("x-bar".into(), "2".into())],
1405 progress: None,
1406 };
1407 let out = upload_options_from(opts);
1408 let map = out.metadata.expect("metadata set");
1409 assert_eq!(map.get("x-foo").map(String::as_str), Some("1"));
1410 assert_eq!(map.get("x-bar").map(String::as_str), Some("2"));
1411 }
1412
1413 // --- from_remote_url constructor branch ---------------------------
1414
1415 #[tokio::test]
1416 async fn from_remote_url_rejects_s3() {
1417 let result = AzureStore::from_remote_url(&s3_url()).await;
1418 match result {
1419 Err(ObjectStoreError::Other(_)) => {}
1420 Err(other) => panic!("expected ObjectStoreError::Other, got {other:?}"),
1421 Ok(_) => panic!("expected S3 URL to be rejected"),
1422 }
1423 }
1424
1425 // --- HTTP transport tuning (#26 / #28) ----------------------------
1426
1427 /// Pin the timeout values. A future copy-paste mistake (`from_millis`
1428 /// instead of `from_secs`, an accidental zero) silently disables
1429 /// the very behaviour these constants exist for; fail fast instead.
1430 /// If the constants are deliberately changed, update the expected
1431 /// values on the right-hand side together — the test exists to make
1432 /// such changes deliberate, not to lock the values forever.
1433 #[test]
1434 fn transport_timeout_constants_have_expected_values() {
1435 assert_eq!(POOL_IDLE_TIMEOUT, Duration::from_secs(30));
1436 assert_eq!(TCP_KEEPALIVE, Duration::from_secs(30));
1437 assert_eq!(CONNECT_TIMEOUT, Duration::from_secs(10));
1438 assert_eq!(READ_TIMEOUT, Duration::from_secs(30));
1439 }
1440
1441 #[test]
1442 fn build_http_client_succeeds() {
1443 build_http_client().expect("reqwest client builds with the configured timeouts");
1444 }
1445
1446 /// The meaningful regression check: if a future refactor drops the
1447 /// `transport = Some(...)` line in `build_client_options`, the
1448 /// Azure backend silently reverts to `azure_core`'s default
1449 /// (unbounded) HTTP transport. This test fails when that happens.
1450 /// Also pins the empty-policies invariant on the no-credential
1451 /// branch, so a refactor that injects a fallback policy when
1452 /// `per_try_policy` is `None` is caught.
1453 #[test]
1454 fn build_client_options_installs_custom_transport() {
1455 let resolved = auth::ResolvedCredentials {
1456 token_credential: None,
1457 per_try_policy: None,
1458 sas_signing_key: None,
1459 };
1460 let opts = build_client_options(&resolved).expect("client options build");
1461 assert!(
1462 opts.transport.is_some(),
1463 "ClientOptions::transport must be Some so the SDK uses our \
1464 pool_idle_timeout / tcp_keepalive client (issue #26)",
1465 );
1466 assert!(
1467 opts.per_try_policies.is_empty(),
1468 "no per-try policy was supplied; the helper must not inject \
1469 a fallback signer of its own",
1470 );
1471 }
1472
1473 /// Issue #28's Notes section explicitly calls out: the per-try
1474 /// signing policy must continue to fire after we install a custom
1475 /// transport. The SDK pipeline runs them independently of the
1476 /// transport, but a future refactor that confuses the two fields
1477 /// would silently drop signing — surface that here. The
1478 /// [`Arc::ptr_eq`] check pins identity so a refactor that
1479 /// silently *replaces* the caller's policy with a fresh one
1480 /// (rather than dropping it outright) also fails.
1481 #[test]
1482 fn build_client_options_preserves_per_try_policy() {
1483 // Azurite's published well-known account key — base64-valid
1484 // and safe to embed.
1485 const AZURITE_KEY: &str = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";
1486 let policy: Arc<dyn azure_core::http::policies::Policy> = Arc::new(
1487 auth::SharedKeySigningPolicy::new("devstoreaccount1", AZURITE_KEY)
1488 .expect("shared-key policy constructs"),
1489 );
1490 let resolved = auth::ResolvedCredentials {
1491 token_credential: None,
1492 per_try_policy: Some(Arc::clone(&policy)),
1493 sas_signing_key: None,
1494 };
1495 let opts = build_client_options(&resolved).expect("client options build");
1496 assert!(opts.transport.is_some(), "transport still wired");
1497 assert_eq!(
1498 opts.per_try_policies.len(),
1499 1,
1500 "exactly one per-try policy is wired",
1501 );
1502 assert!(
1503 Arc::ptr_eq(&policy, &opts.per_try_policies[0]),
1504 "the policy at index 0 must be the same Arc the caller \
1505 supplied — not a fresh policy constructed inside the helper",
1506 );
1507 }
1508
1509 /// Pin the `should_use_multipart` predicate at and around the
1510 /// shared threshold (issue #53).
1511 ///
1512 /// `put_bytes` and `put_path` route through this predicate. The
1513 /// integration test `multipart_put_emits_per_block_progress_events`
1514 /// covers the dispatch *call* (only multipart emits per-block
1515 /// events). This unit test pins the predicate's boundary semantics
1516 /// so the constant can't be moved out from under that test
1517 /// without something failing. The Azure backend uses the same
1518 /// shared `MULTIPART_PUT_THRESHOLD` as S3 so a future refactor
1519 /// cannot accidentally raise the threshold for one backend alone.
1520 #[test]
1521 fn should_use_multipart_pins_threshold_boundary() {
1522 use super::super::multipart::MULTIPART_PUT_THRESHOLD;
1523 assert!(!should_use_multipart(MULTIPART_PUT_THRESHOLD - 1));
1524 assert!(should_use_multipart(MULTIPART_PUT_THRESHOLD));
1525 assert!(should_use_multipart(MULTIPART_PUT_THRESHOLD + 1));
1526 assert!(should_use_multipart(6 * (1 << 30)));
1527 }
1528
1529 /// Pin `block_id_for(idx)` so two parts can never collide on the
1530 /// same block ID, and so all IDs in a single `commit_block_list`
1531 /// share a length pre-base64 (Azure's hard requirement).
1532 #[test]
1533 fn block_id_for_is_unique_and_uniform_length() {
1534 let id_a = block_id_for(0);
1535 let id_b = block_id_for(1);
1536 let id_c = block_id_for(99_999);
1537 assert_eq!(id_a.len(), id_b.len(), "all IDs share length");
1538 assert_eq!(id_a.len(), id_c.len(), "even at the upper end");
1539 assert_ne!(id_a, id_b, "two parts get distinct IDs");
1540 // 32 ASCII bytes accommodates up to 10^32 parts — vastly above
1541 // [`AZURE_MAX_BLOCKS`] = 50 000.
1542 assert_eq!(id_a.len(), 32);
1543 }
1544}