Skip to main content

hearth_core/
lib.rs

1//! hearth — deterministic model residency.
2//!
3//! Keep declared models warm, and tell the truth about which ones are.
4//!
5//! Not an inference engine. llama.cpp and vLLM have spent years on kernels,
6//! samplers and tokenizers, and none of that is the problem. The problem is
7//! that no serving stack will promise a model stays loaded, and none of them
8//! can say why one stopped being. hearth is the supervisor above the runtime:
9//! it declares a set of models, refuses to declare more than the card holds,
10//! keeps them pinned, and reports residency as a named state with a named
11//! reason instead of an eventual timeout.
12//!
13//! Two states here do not exist anywhere else, and they are the two that cost
14//! a night of debugging to tell apart:
15//!
16//!   * `Lost { reason: Evicted }`     — the runtime dropped it for VRAM.
17//!   * `Lost { reason: GpuDetached }` — the host took the card away.
18//!
19//! One is a capacity problem you own. The other is your provider's, and no
20//! configuration you write will fix it. Today both look identical from the
21//! outside, which is why both get "fixed" repeatedly and neither goes away.
22
23pub mod budget;
24pub mod fleet;
25pub mod gguf;
26pub mod residency;
27pub mod sha256;
28
29pub use budget::{plan, total_ctx_tokens, Budget, Declared, Plan, Rejection, GIB};
30pub use fleet::{Fleet, Route, Slot};
31pub use gguf::{read_kv_shape, KvShape};
32pub use residency::{LostReason, Millis, Observation, Residency};