Module cloud_fetch 

The cloud-provider range lifecycle: the fetch payload parsers, the Azure ServiceTags URL extraction, the refresh cadence with the single-flight gate, and the store-backed cache layers.

This is the pure half of the reference cloud handler (guard_core/handlers/cloud_handler.py, _cloud_provider_fetchers.py, _cloud_azure_fetch.py, _cloud_provider_registry.py, cloud_ip_stores.py); the wire half - the HTTP client driving the reference endpoints - lives in the facade’s cloud_fetch and feeds these parsers.

Payload parsers (one per provider, the reference fetcher bodies)

AWS    json prefixes[]: entries with service == "AMAZON" only; the
       ip_prefix network carries its region annotation; a malformed
       prefix aborts the whole parse (the reference raise)
GCP    json prefixes[]: ipv4Prefix or ipv6Prefix, scope annotation;
       a malformed prefix aborts the whole parse (the reference raise)
CSV    `DigitalOcean` / Linode: the first field per non-comment line;
       invalid CIDRs are skipped, not fatal
Vultr  json subnets[]: ip_prefix; invalid CIDRs are skipped
Azure  json values[]: the "AzureCloud" entry's
       properties.addressPrefixes; a malformed prefix fails the whole
       parse (the reference raises into the fetcher's catch)

Azure ServiceTags URL extraction (the reference order)

1. the id="failoverLink" anchor’s href;
2. the newest URL matching the ServiceTags pattern (newest = the ServiceTags_Public_(\d{8}) date, then the URL text);
3. any https://download.microsoft.com/...json href.

Every candidate must be https on download.microsoft.com, or it is rejected.

Refresh lifecycle

CloudRefreshCoordinator carries the reference stamp semantics: the check asks CloudRefreshCoordinator::should_refresh (now - last > interval, the reference CloudIpRefreshCheck), stamps optimistically, schedules, and rolls the stamp back when scheduling reports failure. CloudRefreshCoordinator::schedule_refresh is single-flight: while one refresh is in flight, further calls are no-ops returning false, so at most one refresh runs at a time.

refresh_provider_from_store walks the reference cache layers, newest-to-oldest: the store (the section 08 cloud_ip_v2 / cloud_ranges_v2 shapes) first - a hit decodes and installs in memory with no network fetch; a miss fetches, and only a non-empty result is cached and installed (an empty fetch is never cached and never touches the provider’s previous ranges).

Example

use guard_core_engine::cloud_fetch::{parse_aws_ranges, extract_azure_download_url};

// The AWS shape: only the AMAZON service entries, regions attached.
let payload = r#"{"prefixes": [
  {"ip_prefix": "203.0.113.0/24", "region": "us-east-1", "service": "AMAZON"},
  {"ip_prefix": "198.51.100.0/24", "region": "eu-west-1", "service": "EC2"}
]}"#;
let ranges = parse_aws_ranges(payload).expect("valid payload");
assert_eq!(ranges, vec![(String::from("203.0.113.0/24"), Some(String::from("us-east-1")))]);

// The Azure page scrape: the newest `ServiceTags` URL wins, on-host only.
let page = r#"<a id="failoverLink" href="https://download.microsoft.com/?x">x</a>"#;
assert_eq!(
    extract_azure_download_url(page).as_deref(),
    Some("https://download.microsoft.com/7/1/ServiceTags_Public_20260101.json")
);

Structs

AzureDownloadCandidate

A selected ServiceTags URL and how fresh the reference considers it.

CloudRefreshCoordinator

The refresh stamp plus the single-flight gate (the reference CloudIpRefreshCheck stamp and CloudManager.schedule_refresh).

PayloadError

A malformed provider payload: the JSON body did not carry the shape the parser reads.

Enums

CloudRefreshOutcome

What one refresh_provider_from_store pass decided.

Constants

AZURE_TRUSTED_DOWNLOAD_HOST

The trusted Azure download host (_AZURE_TRUSTED_DOWNLOAD_HOST): every candidate URL must be https on this exact host.

Traits

CloudRangeStore

The store seam behind the cache layers (the reference CloudIpStoreProtocol; the Redis shape is the section 08 cloud_ip_v2 / cloud_ranges_v2 payload, the in-memory shape is a map).

Functions

extract_azure_download_url

The full extraction (_extract_azure_download_url): failover link, then the newest ServiceTags URL, then any trusted JSON href.

extract_failover_link_url

The failover-link arm (_extract_failover_link_url): the first <a ... id="failoverLink" ...> anchor’s href, trusted URLs only.

extract_generic_json_url

The generic-json arm (_extract_generic_json_url): the first https://download.microsoft.com/...json href.

extract_newest_service_tags_url

The newest-ServiceTags arm (_extract_newest_service_tags_url): every URL matching the ServiceTags shape on the trusted host, the winner by (has date, date, url).

is_trusted_azure_download_url

Whether url is a trusted Azure download URL (_is_trusted_azure_ download_url): https scheme on the exact trusted host.

parse_aws_ranges

The AWS parser (fetch_aws_ip_ranges): the prefixes[] entries with service == "AMAZON" only, each ip_prefix carrying its region as the annotation.

parse_azure_service_tags

The Azure ServiceTags parser (_select_azure_cloud_prefixes): the values[] entry named AzureCloud, its properties.addressPrefixes.

parse_csv_ranges

The CSV parser (_fetch_csv_prefix_networks, the DigitalOcean and Linode shape): the first field of every non-comment line; blank lines are skipped and an invalid prefix is skipped, not fatal.

parse_gcp_ranges

The GCP parser (fetch_gcp_ip_ranges): the prefixes[] entries’ ipv4Prefix / ipv6Prefix with the scope annotation.

parse_service_tags_date

The ServiceTags_Public_(\d{8}) date of url (_parse_service_tags_date): None when the pattern is absent, the digits do not form a real calendar date, or the date lies in the future.

parse_vultr_ranges

The Vultr parser (fetch_vultr_ip_ranges): the subnets[] entries’ ip_prefix; an invalid prefix is skipped, not fatal.

refresh_provider_from_store

One provider’s refresh pass over the reference cache layers: the store first (a hit installs, no fetch), then the fetcher (only a non-empty result is cached and installed).

Type Aliases

ParsedRanges

The parsed provider ranges: (network, region) pairs in the crate::cloud_provider::CloudIpTable entry shape.