Expand description
costroid-power — the local-inference economics engine (the headline differentiator, §3.1.B).
This crate owns the dual-mode power/cost model: it turns a local model run (token counts + real or estimated power draw integrated over wall-clock time) into joules/token, Wh per 1M tokens, and $ per 1M tokens, with the measurement source stamped on every figure (R6).
§Status — M3a in progress (built on the M0 scaffold)
In place and green on every target: the PowerSampler abstraction (§6.3) with its four
sources and the wall-meter-led runtime selector, the MeasurementMode stamp (§6.4), and
the cost model (§3.2) with deterministic worked-example tests. Deferred to M3b
(human-gated, on real hardware): the live sysfs read confirmation on the gfx1151 APU, the
LibreHardwareMonitor live loopback read (M3a ships its parser seam), and a captured
joules/token figure — no real power number is ever fabricated in code (R10).
§Invariants honored here
- No
unwrap/expect/panic!in library code; every failure is aPowerError. - Gated by
#[cfg(target_os = "linux")]+ the off-by-defaultpowerfeature (R1, namedpowernottelemetry): non-Linux / feature-off builds compile a clean “unavailable” path, never a broken build, and the default workspace build never links a power path. - No network, no GPU FFI: the inference runner is a subprocess (A2), so this crate
pulls no HTTP client and (with
unsafe_code = "forbid"workspace-wide) nounsafebindings.
Re-exports§
pub use cost::cloud_cost;pub use cost::local_run_cost;pub use cost::CostInputs;pub use cost::LocalRunCost;pub use harness::average_watts;pub use harness::benchmark_id;pub use harness::compute_report;pub use harness::estimate_run;pub use harness::run_measured;pub use harness::LocalRunReport;pub use harness::DEFAULT_BENCHMARK_SUITE;pub use models::bundled_models;pub use models::ModelManifest;pub use models::ModelQuality;pub use models::ModelSpec;pub use models::GEMMA4_MANIFEST_AS_OF;pub use profile::bundled_power_profiles;pub use profile::ElectricityRate;pub use profile::HardwareProfile;pub use profile::PowerProfiles;pub use profile::ProfileOverrides;pub use profile::ResolvedProfile;pub use profile::DEFAULT_HARDWARE_PROFILE_ID;pub use profile::POWER_PROFILE_AS_OF;pub use runner::parse_llama_cpp_stats;pub use runner::parse_ollama_stats;pub use runner::LlamaCppRunner;pub use runner::OllamaRunner;pub use runner::RunOutput;pub use runner::RunSpec;pub use runner::Runner;pub use runner::StubRunner;
Modules§
- cost
- The local-inference cost model (§3.2) — implemented exactly, with deterministic tests.
- harness
- The benchmark harness (M3) — turns a local run (token counts + power over wall-clock) into the §3.2 economics: Wh, J·token⁻¹, and $·(1M tok)⁻¹, with the measurement mode stamped (R6).
- models
- The bundled, dated Gemma 4 model manifest (M3, R10).
- profile
- The bundled, dated, overridable power/cost assumptions (M3, R8).
- runner
- The local-inference subprocess runner (M3, A2).
Structs§
- Estimated
Power Sampler - Derives power from a transparent profile: load draw scaled by utilization. The universal fallback, always available, and the basis for what-if scenarios (D/§3.1).
- Sysfs
Power Sampler - Reads
power1_averagefrom an amdgpu hwmon node (µW → W). Real readings are compiled in only under#[cfg(all(target_os = "linux", feature = "power"))]; on every other target, and whenever thepowerfeature is off, it compiles to a stub whoseprobe()isfalseand whosesample_watts()returnsPowerError::SensorUnavailable— so the type always exists, the build is green everywhere, and no command is ever removed (R1). - Wall
Meter Power Sampler - True total-system power from a user-configured external meter. The M0 scaffold models the simplest source — a constant manual value (watts) the user supplies; the CSV/log-feed and smart-plug-local-API variants (still blocking I/O, never an async HTTP client — D) land at M3. Works on every OS, so measured energy stays available from WSL2/Windows.
- Windows
LhmPower Sampler - Reads the SMU Package power sensor from a running LibreHardwareMonitor’s separate-process
JSON endpoint (
http://localhost:8085/data.json). MPL-2.0 stays cleanly out-of-process — no linking, no FFI, no .NET in our binary (§5.4) — and the read is a blocking local-loopback read, not a network egress.
Enums§
- Measurement
Mode - How a power reading was obtained. Ordered most- to least-authoritative along the revised (2026-06-20) wall-meter-led ladder (§5.3/§5.4): the wall meter is the truest cross-OS figure, the on-chip readers (sysfs on Linux, LibreHardwareMonitor on Windows) are package-grade, estimated is the universal fallback.
- Power
Error - Anything that can go wrong while sampling power or computing local-inference economics.
Constants§
- MEASURED_
MODELS - The authoritative, human-gated allowlist of model ids whose committed
benchmarks/**dataset carries a real measured run — paired with the exactMeasurementModethat run used.
Traits§
- Power
Sampler - A source of instantaneous power draw, in watts.
Functions§
- measured_
mode_ for - The declared measured mode for
model_id, if it is on the authoritativeMEASURED_MODELSallowlist;Nonefor any model still estimated (every model, pre-M3b). - select_
sampler - Choose the active power source along the revised (2026-06-20) wall-meter-led ladder
(§5.3/§5.4, D1): a configured wall meter wins (it is the truest cross-OS figure); else the
on-chip package-grade source available on this host (
sysfson native Linux, then LibreHardwareMonitor on Windows); else the estimated fallback (always available). The selected sampler’sPowerSampler::modeis what gets stamped onto every record.