Expand description
One OpenAI-compatible port in front of a hearth fleet.
The proxying is the boring part. What makes this worth having is that when
hearth cannot serve a request, the HTTP response says which of four very
different things happened — still loading, evicted, GPU reclaimed, or will
never fit — and whether retrying here could ever work. See gateway.
Re-exports§
pub use gateway::decide;pub use gateway::Decision;pub use gateway::Endpoint;pub use gateway::Response;pub use http::Request;pub use http::Server;