mock_upcloud/

lib.rs

//! **`mock-upcloud` — the UpCloud API 1.3, faithfully wrong.**
//!
//! A real HTTP server that answers UpCloud's 1.3 surface out of a state machine.
//! Point `UPCLOUD_API_BASE` at it and the real `monetize-cloud-impl` upcloud
//! plugin, the real `gunnar/deploy/upcloud` procedure and the real `cargo xtask`
//! estate verbs talk to it without knowing. Nothing here reaches the network.
//!
//! # The point is the lies, not the happy path
//!
//! A mock that answers every call correctly proves nothing, because that is not
//! the provider anybody has. Every behaviour below was MEASURED against the live
//! account or is written down in this estate's history (3, 5 and 37 are the
//! exception: verified against Scaleway, not UpCloud — see below), and each one is armable
//! on its own ([`faults::Fault`]) so a defect can be reproduced on demand instead
//! of waited for.
//!
//! | # | behaviour | where it is modelled |
//! |---|---|---|
//! | 1 | a firewall-rule read for a DELETED server answers `403 ERROR_AUTHENTICATION_FAILED`, not 404 | [`http::firewall_rule`], always on — it is not a fault, it is the API |
//! | 2 | `GET /1.3/price` fails at the TRANSPORT level, not with a status | [`Fault::PriceTransportReset`] — the socket is closed mid-reply |
//! | 3 | `412 out_of_stock` on poweron, for days — **verified against Scaleway (fr-par-1 2026-09-09), not UpCloud; kept as a generic cloud-provider fault** (UpCloud documents `409 STORAGE_RESOURCES_UNAVAILABLE`) | [`Fault::OutOfStock`], sticky until disarmed |
//! | 4 | the server LIST carries no `storage_devices`; the storage list DOES carry `labels` | [`render`], always on |
//! | 5 | a revoked credential answers 0 servers and 403 on some endpoints, never a clean 401 — **verified against Scaleway (IAM change 2026-08-26), not UpCloud; kept as a generic cloud-provider fault** (a dead UpCloud token is a clean 401: 43) | [`Fault::RevokedCredential`] |
//! | 6 | create a server: 98–105 s | [`clock::Timings::server_create`] |
//! | 7 | delete front/twin ≈ 60 s; delete an appliance carrying 4 member volumes > 5 min in `maintenance` | [`clock::Timings::server_delete`] |
//! | 8 | the guest's own install ≈ 10 s (`INSTALL-OK` 10 011 ms measured under KVM) | [`clock::Timings::guest_install`] / the KVM half |
//! | 9 | `stop-resize-start` leaves an auto-titled `Resize Backup`, detached, labelled, WITHOUT "gunnar" in the title | [`estate::Estate::modify_server`] |
//! | 10 | ~~young objects are indistinguishable from orphans without `created`~~ **WRONG, and corrected below** | `created` is SENT, on both the list and the detail; [`Fault::WithholdCreatedField`] takes it away on request |
//! | 11 | a server must be stopped (hard) before it can be deleted with its storages | [`estate::Estate::delete_server`] |
//! | 12 | utility IPs are reassigned between re-lays; public IPs change every time | [`estate::Addresses`], shuffled per lay from the seed |
//! | 13 | no serial console — VNC only; a PID-1 panic is invisible except on the framebuffer | the KVM half boots `-serial none` |
//! | 14 | the VNC port goes stale on every stop/start and the API keeps reporting the old one | [`Fault::StaleVncPort`] (default ON — it is the provider's normal) |
//! | 15 | SeaBIOS, no UEFI | the KVM half refuses an OVMF spec by name |
//! | 16 | `cdrom/eject` works on a STARTED server; IDE/virtio detach does not (the virtio half is the guest's — see 60) | [`estate::Estate::eject`] / [`estate::Estate::detach`] |
//! | 17 | a re-image killed mid-install loops the installer (CD stays first in the boot order) | [`estate::Guest::boot`] |
//! | 18 | inbound UDP replies are dropped (DNS/NTP) | [`Fault::UdpInboundDropped`], default ON; [`guest_clock::ntp_answer`] answers silence |
//! | 19 | a guest boots one TIMEZONE OFFSET ahead — +2 h in summer, +1 h in winter — and not every guest does | [`guest_clock`]: the hypervisor's RTC is correct UTC, the GUEST misreads it ([`Fault::GuestReadsRtcAsLocalTime`]) |
//! | 20 | the installer PID 1 brings no network up, so nothing narrates the install window | the KVM half's install image |
//! | 21 | on first boot `unattended-upgrade-shutdown --wait-for-signal` holds `dpkg lock-frontend` for the life of the boot | the Ubuntu template guest |
//! | 22 | a volume NEVER shrinks; a growth is permanent and billed forever | [`estate::Estate::modify_storage`] refuses a shrink by name |
//! | 23 | the direct-upload import and the storage have **two different clocks**: the import says `completed` five seconds in, the storage sits in `syncing` for 100–130 s more | [`estate::Estate::upload`] → [`estate::After::BeginSync`] |
//! | 24 | the import object carries `read_bytes`/`written_bytes` and REAL `md5sum`/`sha256sum` — the ladder verifies the sha256 against the local file | [`digest`], which computes them rather than inventing them |
//! | 25 | the same 43 MB medium is uploaded TWICE per re-image (cdrom, then virtio, because `korp-installer` probes virtio alone) — two imports, two syncs, ~4 min of pure waiting | [`clock::Timings::storage_sync_lo_ms`] |
//! | 26 | `POST /storage/{uuid}/clone` is the candidate fix, and whether it skips the sync is **MEASURED after all — see 53** | [`estate::Estate::clone_storage`]; pessimistic by default, [`Fault::CloneSyncsLikeImport`] for the optimistic guess |
//! | 27 | one wrong clock is SIX red rows: skew-refusal → no-quorum → clock-unchanged → console-key → banner → monetize-poll | [`guest_clock::cascade`] |
//! | 28 | **the skew window is never widened**: a row that needs a wider window is a RED | [`guest_clock::widening_the_window_still_leaves_a_red`] — no number turns a wrong clock green |
//! | 29 | **DHCP option 121 is offered and a guest that ignores it has NO ROUTE off its /22** — outbound dies, inbound stays healthy | [`net::outbound_reach`] / [`Fault::GuestIgnoresDhcpOption121`] |
//! | 30 | **hairpin NAT does not exist**: the box holding a DNAT cannot use it on its own public address | [`net::Reach::NoHairpin`] — and it reads `connection refused` |
//! | 31 | the console is a HOST and a port, and the toggle re-provisions BOTH (`se-sto1.vnc.upcloud.com:60031`) | [`estate::Server::console`], [`render::server`] |
//! | 32 | the guest's install is 2–5 s on the UART while the ladder's wall measurement of the same re-image is 110–253 s | [`clock::Timings::reimage_observers`] — two observers, two orders of magnitude |
//! | 33 | a re-image mints a NEW SSH host key every time; three paths must agree to tell that from a hijacked name | [`estate::Estate::host_key_via`] / [`Fault::HijackedName`] |
//! | 34 | **the door that actually leaks**: `POST /storage/{uuid}/resize` (the twin's xfs growth) takes a `Resize Backup` FIRST, the size of the volume, and hands it back as `resize_backup`; nothing deletes it. Labelled by default (measured); [`Fault::ResizeBackupUnlabelled`] is the brief's hypothesis that it carries nothing at all | [`estate::Estate::resize_filesystem`] |
//! | 35 | the firewall of a server that READS FINE answers the same `403 ERROR_AUTHENTICATION_FAILED` as behaviour 1 — a credential scoped without the firewall permission; only the server's own detail tells "forbidden" from "gone" | [`Fault::FirewallForbidden`], by name only (INFERRED, not measured) |
//! | 36 | `GET /1.3/server/{uuid}` answers `404 SERVER_NOT_FOUND` for a uuid the LIST carries — the account contradicting itself | [`Fault::DetailNotFoundForListedServer`], by name only (REPORTED 2026-09-20, uuids never compared before the box was wiped) |
//! | 37 | a sold-out zone refuses the **create** with the same `412 out_of_stock` it refuses the poweron with, and mints nothing — **verified against Scaleway (fr-par-1 2026-09-09), not UpCloud; kept as a generic cloud-provider fault** | [`estate::Estate::create_server`] / [`Fault::OutOfStock`], one sticky fault for both doors |
//! | 38 | a detach answers `200` **and the volume stays attached** — the write that reports its own success and did not happen; only a read-back tells it from a real one | [`estate::Estate::detach`] / [`Fault::DetachSaysSuccessButStaysAttached`], by name only (REPORTED) |
//! | 39 | an empty path segment (`/1.3//storage`) is a `404` that names no path | [`http`], always on — gunnar `api.rs`, MEASURED |
//! | 40 | a READ answers `502` now and then; a poll must survive it | [`Fault::ReadBadGateway`], by name (REPORTED, gunnar `wait.rs`) |
//! | 41 | `PUT /1.3/server/{uuid}` answers `202` | [`http`], always on — MEASURED 2026-09-19 |
//! | 42 | a chunked PUT to the direct-upload URL answers `411` | [`http`], always on (REPORTED, gunnar `http.rs`) |
//! | 43 | a DEAD token is a clean `401 AUTHENTICATION_FAILED` on every call | [`Fault::DeadToken`] — MEASURED 2026-09-14 (ROTATION §2.3) |
//! | 44 | past the account's MaxIOPS quota (yvra: 10 240 GiB) a create or grow is `403 MAXIOPS_STORAGE_LIMIT_REACHED` | [`estate::Estate::create_storage`] / `modify_storage`, always on (limit MEASURED, code REPORTED) |
//! | 45 | a disk boot reads `started` ~10 s after the start (was 20 s here) | [`clock::Timings::start_ms`] — MEASURED 2026-09-19 |
//! | 46 | **an installer that powers the box off never reads `started`**: `maintenance` t+10…t+130 s, then `stopped` | [`clock::Timings::install_pass_lo_ms`], [`estate::After::GuestPoweredOff`] — MEASURED 2026-09-15/19 |
//! | 47 | `stop_server` with `timeout_action` is a `400` (it belongs to restart) | [`http`], always on — MEASURED 2026-09-14 |
//! | 48 | a plan change is done when the PUT answers (the provider starts at once); a `start` during `maintenance` is `409` | [`estate::Estate::modify_server`] / `start_server` — the provider's order MEASURED 2026-09-21 |
//! | 49 | a `boot_order` PUT does not end an install loop; it is read at the next HYPERVISOR start | always on, now tested — MEASURED 2026-09-14 |
//! | 50 | the VNC toggle settles ~2 s after the "no"; a "yes" inside the window gives the OLD endpoint back | [`clock::Timings::vnc_settle_ms`] — MEASURED (gunnar `console.rs`) |
//! | 51 | the console toggle is `409` while the server is `maintenance` | [`estate::Estate::modify_server`] — MEASURED (memory: upcloud-vnc-port-toggle) |
//! | 52 | an import's storage ends `syncing` → `maintenance` → `online` | [`clock::Timings::sync_tail_maintenance_ms`] — sequence MEASURED, split not |
//! | 53 | **a clone is `maintenance` → `online` in 47 s with NO `syncing`** (26's "not measured" was measured) | [`clock::Timings::clone_online_ms`]; the old guess is [`Fault::CloneSyncsLikeImport`] — MEASURED 2026-09-20 |
//! | 54 | a shrink AND a same-size PUT are `400 SIZE_INVALID` "The new size must be greater than the old size." | [`estate::Estate::modify_storage`] — MEASURED 2026-09-14 (22's code was wrong) |
//! | 55 | a size grow is `maintenance` for 37 s, then `online` | [`clock::Timings::storage_grow_ms`] — MEASURED 2026-09-14 |
//! | 56 | a grow of a storage attached to a RUNNING server is `409 STORAGE_ATTACHED` (terraform 5.44.1 sends it without stopping) | always on; [`Fault::ResizeRequiresDetach`] for the stricter docs page — REPORTED (docs) |
//! | 57 | a storage already attached is `409 STORAGE_ATTACHED` on a second attach | [`estate::Estate::attach_at`] — REPORTED |
//! | 58 | a requested address is honoured, `virtio` takes the first FREE slot, and the guest names virtio disks by slot order, contiguously (vda = virtio:0) | [`estate::Estate::attach_at`], [`estate::Estate::guest_disk_names`], `/mock/disks/{uuid}` |
//! | 59 | a storage in `maintenance`/`syncing`, or the source of a clone still being made, cannot be deleted: `409 STORAGE_STATE_ILLEGAL` | [`estate::Estate::delete_storage`] — REPORTED |
//! | 60 | **virtio hot-plug is the GUEST's**: a hot-plug kernel (needs `PCI_MSI`) takes an attach/detach on a running server; one without answers `511 HOTPLUG_FAILED` (16's `409 VIRTIO_HOTPLUG_UNSUPPORTED` was invented) | [`Fault::GuestKernelLacksHotplug`], per server at create — MEASURED 2026-09-14 |
//! | 61 | an interface that asks for `IPv6` gets IPv6 ("no IPv6" is the estate's law, not UpCloud's) | [`render::server`] — REPORTED |
//! | 62 | **the firewall is evaluated**: off or no rules = open; top-down first match; a drop is silence; the utility net is filtered; UDP replies die behind a catch-all drop | [`net::firewall_admits`], `/mock/inbound/{uuid}`, `/mock/udp-reply/{uuid}` — MEASURED 2026-09-13/14/20 |
//! | 63 | the mock says whom it heard (per credential digest), so a terraform verb can prove the provider talked to THIS mock before `apply`/`destroy` | [`http::Mock::heard_from`], `/mock/heard?token_sha256=` |
//! | 64 | a server PUT changes `hostname`/`title` in place (the mock dropped both; terraform refused its own apply) | [`estate::Estate::rename_server`] — MEASURED 2026-09-21 through 5.44.1 |
//! | 65 | `POST /server/{uuid}/cdrom/load` puts a medium into an EMPTY tray; a loaded one is `409 CDROM_DEVICE_IN_USE` | [`http`] — L110, DOCUMENTED, not measured |
//! | 66 | an import straight after `POST /storage` (no wait) is accepted; any other non-online state is still refused | [`estate::Estate::start_import`], [`estate::After::Created`] — acceptance MEASURED (live receipt 2026-09-21, gunnar `plan.rs:550`); the state it read then is not recorded |
//! | 67 | the direct-upload `PUT` answers the import object BARE (`{"written_bytes": …, "sha256sum": …}`), not wrapped in `storage_import` | [`http`] — MEASURED: the live re-image of 2026-09-21 08:42 read it at the top level, `done: true` |
//! | 68 | **the verified spellings**: a storage `size` is sent as a JSON NUMBER (verified live, monetize grow 2026-09-14), a stop `timeout` as a STRING (verified live, gunnar-upcloud + monetize); the other spelling of each is UNVERIFIED, so the mock accepts both rather than invent an answer | [`http`] — ledger L112 |
//!
//! Rows 39–63 are lane T13's, from `BEHAVIOURS-LEDGER.md` beside this crate,
//! where every one carries its source. Three rows above are **not UpCloud's**:
//! 3 and 37 (`412 out_of_stock`) and 5 ("0 rows + 403") were verified against
//! Scaleway (fr-par-1 2026-09-09; the IAM change 2026-08-26), not UpCloud.
//! Owner ruling 2026-09-21: they are kept as generic cloud-provider faults —
//! relabelled, not removed, because removing them would re-map every storm
//! seed — and they are never to be read as UpCloud behaviour.
//!
//! Behaviours 35 and 36 exist for ONE caller: `private-gunnar-ops`'s
//! `estate::prune_dead_firewalls` and the row F38 that proves it. Together with
//! 1 and 5 they are the four ways a `403`/`404` pair can lie about whether a
//! server is there, and a caller that reads the list, the detail AND the
//! firewall before it drops a state entry is the only caller that survives all
//! four. A stale state naming a server that was re-laid under a NEW uuid — the
//! likeliest reading of 2026-09-20 — needs no fault at all: create the estate,
//! ask about a uuid that was never created, and the mock answers 404 on the
//! server and 403 on its firewall while the list shows the same titles alive.
//!
//! # Two callers, one API
//!
//! Everything above is the surface `monetize-cloud-impl`'s plugin and
//! `gunnar/deploy/upcloud` drive. `private-holger-ops` drives the SAME account
//! through terraform and `UpCloudLtd/upcloud` 5.44, which carries
//! `UPCLOUD_DEBUG_API_BASE_URL` — so the real declaration and the real provider
//! can be pointed here too, and holger's twin, which has never been created
//! because creating it costs money, can be laid down for nothing. That door is
//! [`tf`], and the first thing it proved was about this crate rather than about
//! holger: a device's `storage_size` was rendered as a STRING here, the
//! official provider's own SDK declares it `int`, and every terraform server
//! create died on a shape the account cannot send. Behaviour 10's lesson, a
//! second time.
//!
//! # The calibration
//!
//! Every fault above is a claim that the mock can be wrong in a particular way,
//! and a claim nobody has checked is a claim. [`self_check`] drives EVERY
//! variant in [`Fault::ALL`] twice — disarmed and armed — and fails if the two
//! observations are the same, because a fault that changes nothing a caller can
//! see advertises a coverage this crate does not have. `mock-upcloud
//! --self-check` prints the table and exits non-zero. Behaviour 38 is what it
//! is for: `Estate::detach` had no path that answered `200` without removing
//! the device, so every sweep driven against this mock passed a test the mock
//! could not fail.
//!
//! # Time
//!
//! Every duration above is real, and running 100 000 of them at real speed is
//! three years. So the mock keeps a **virtual clock** ([`clock::Clock`]) with a
//! `speed` knob: `1.0` is faithful wall time, `0.0` makes every transition
//! instant, `100.0` is a hundred times quicker. The ORDER and the RELATIVE
//! durations never change — a storage still goes `maintenance` → `maintenance` →
//! `online`, a delete still traverses `maintenance`, and a client that does not
//! poll still fails. What changes is only how long the harness waits.
//!
//! This is the reason the mock is a state machine over a clock rather than a
//! table of canned replies: a canned reply cannot be *early*.
//!
//! # Faults
//!
//! [`faults::Faults`] is a set. A fault can be armed by name (deterministic: it
//! fires every time it can) or a **seed** can be handed in, in which case the
//! fault weather for the whole run is derived from that seed alone. A failing
//! run is replayable from its seed and nothing else — no recording, no capture
//! file. See [`rng::SplitMix64`]: the generator is thirty lines and in this
//! crate on purpose, because a dependency's PRNG may change its stream between
//! versions and then a seed stops naming a run.
//!
//! # Behaviour 10 was wrong, and the correction is the more useful entry
//!
//! This crate shipped claiming the API sends no `created` on storage rows, so a
//! young volume and an old orphan were the same row. That was the brief and the
//! brief was mistaken. MEASURED against the live account 2026-09-20 — two
//! volumes, both endpoints, `created` present every time:
//!
//! ```text
//! 018a7b17-5698-4aa9-b160-f6a6daea9921
//!   GET /1.3/storage/private  → "created": "2026-09-20T10:35:18Z"
//!   GET /1.3/storage/{uuid}   → "created": "2026-09-20T10:35:18Z"
//! the Resize Backup's detail  → "created": "2026-09-19T21:54:06Z"
//! ```
//!
//! **A mock stricter than the provider is worse than one that is laxer.** This
//! crate wrote that sentence itself, about the virtio hot-plug attach it used to
//! refuse and which manufactured 82 % of a storm's failures. The same mistake,
//! twice, in opposite directions: a lax mock misses a bug, a strict one invents
//! a verdict path the account cannot reach. It is written here rather than
//! quietly fixed because the correction is worth more than the entry was.
//!
//! What remains true is smaller and still worth having: the `Resize Backup`
//! carries no product name in its title (behaviour 9), so a TITLE filter is
//! blind to it — and that is what `labelled_gunnar` in `private-gunnar-ops`
//! now exists for. The date was never the missing half; the label was.
//!
//! The storage rows the mock sends carry `created` because that is MEASURED.
//! The server rows carry it too, which is INFERRED from the same API sending it
//! for storages and has not been measured on its own — marked here rather than
//! presented as fact, and erring toward sending it for the reason above.
//!
//! # What it is NOT
//!
//! It is not a simulator of UpCloud's billing, its IP transfer market, its
//! network products (routers, gateways, load balancers, object storage), its
//! managed databases or its Kubernetes. It answers the subset this estate
//! drives, listed in [`http`], and answers `404` with a `NOT_IMPLEMENTED`
//! envelope naming the path for everything else — loudly, so an unimplemented
//! call can never read as a working one.

pub mod clock;
pub mod digest;
pub mod estate;
pub mod faults;
pub mod guest_clock;
pub mod http;
pub mod net;
pub mod render;
pub mod tf;
pub mod rng;
pub mod self_check;
pub mod kvm;
pub mod face;

pub use clock::{Clock, Timings};
pub use estate::Estate;
pub use faults::{Fault, Faults};
pub use http::{serve, Mock};
pub use self_check::{self_check, Report, Verdict};