Skip to main content

ServerState

Struct ServerState 

Source
pub struct ServerState { /* private fields */ }
Expand description

Cloneable shared state passed to all server transports.

Implementations§

Source§

impl ServerState

Source

pub async fn build(config: ServerConfig) -> Result<Self, ServerError>

Build shared state from operator configuration.

§Errors

Returns ServerError if the store cannot connect or the engine cannot be constructed.

Source

pub async fn build_with_store<S>( store: S, runtime: RuntimeConfig, ) -> Result<Self, ServerError>

Build shared state from an already-constructed store.

§Errors

Returns ServerError::EngineCall if the engine cannot be constructed.

Source

pub fn from_parts( namespace_resolver: NamespaceResolver, runtime: RuntimeConfig, ) -> Self

Build shared state from explicit parts with a default worker registry.

Source

pub fn from_parts_with_namespace_store<S>( namespace_resolver: NamespaceResolver, runtime: RuntimeConfig, store: Arc<S>, ) -> Self

Build shared state from explicit parts with one caller-supplied durable namespace and worker-deployment leaf.

Identical to Self::from_parts except both control-plane dimensions are derived from store: namespace registry reads/writes and durable worker deployments use the same supplied leaf rather than fresh in-memory state.

Source

pub fn from_parts_with_control_stores( namespace_resolver: NamespaceResolver, runtime: RuntimeConfig, namespace_store: Arc<dyn NamespaceStore>, worker_deployment_store: Arc<dyn WorkerDeploymentStore>, ) -> Self

Build shared state with caller-supplied namespace and worker-deployment stores.

This is the explicit embedder/test seam for retaining both control-plane contracts from one durable leaf without constructing the full engine boot.

Source

pub fn from_parts_with_registry( namespace_resolver: NamespaceResolver, runtime: RuntimeConfig, worker_registry: ConnectedWorkerRegistry, ) -> Self

Build shared state from explicit parts with a caller-supplied registry.

Source

pub fn namespace_guard(&self) -> &NamespaceGuard

Borrow the namespace guard shared by all transports.

Source

pub fn deploy_guard(&self) -> DeployGuard

Build the deploy authorization guard over the shared resolver.

Source

pub fn runtime_config(&self) -> &RuntimeConfig

Borrow non-secret runtime settings needed by transports.

Source

pub fn workspace_root(&self) -> &WorkspaceRoot

Borrow the server-resolved workspace root declared bodies expand {workspace_root} with (#139).

The startup banner reads this so composition points learn the value from the server that will use it, rather than re-deriving it. That banner claim holds for Self::build-constructed states, where this same value is threaded into the declared-body dispatcher; the from_parts* constructors build no such dispatcher, so their copy is inert — readable, but expanded by nothing.

Source

pub fn worker_registry(&self) -> &ConnectedWorkerRegistry

Borrow the connected-worker registry shared by worker transports and dispatch.

Source

pub fn cancel_in_flight_activities( &self, workflow_id: &WorkflowId, ) -> Result<InFlightCancellation, ServerError>

Stop every in-flight activity of workflow_id, by whichever path is executing it (#233).

Joins the three pieces of live state this node holds — the heartbeat tracker (which worker holds which activity), the connected-worker registry (how to reach that worker), and the declared-attempt registry (which commands this server is running itself) — so the cancel handler needs only a ServerState and never learns the routing itself.

Returns one record per tracked in-flight activity, INCLUDING the ones that could not be asked, plus the server-executed declared bodies that were signalled. A worker cancel is a request, not a guarantee; a declared body’s signal reaches its process group.

§Errors

Returns ServerError::LockPoisoned when the tracker’s, the registry’s, or the declared-attempt registry’s state cannot be read.

Source

pub fn declared_attempts(&self) -> &DeclaredCommandAttempts

Borrow the registry of declared bodies this server is executing.

The declared-body dispatcher registers each attempt here for the life of its command; Self::cancel_in_flight_activities signals through it.

Source

pub fn cluster_publisher(&self) -> &ClusterEventPublisher

Borrow the WS3 cluster-event publisher shared by the cluster state-change sites (supervisor, worker registry) and the cluster subscription endpoint. Always present, on every boot.

Source

pub fn transcript_publisher(&self) -> &ActivityEventPublisher

Borrow the NOI-5b transcript sequencer shared by the worker->server ingestion seam (which publishes a running activity’s ActivityEvents) and the transcript subscription endpoint (which tails + resumes them). Always present, on every boot.

Source

pub fn attempt_owners(&self) -> &AttemptOwnerIndex

Borrow the NOI-6 attempt -> owning-worker back-index. The agent-dispatch path binds an owner when it dispatches an agent attempt and releases it on completion, so the intervention router always resolves the CURRENT owner.

Source

pub fn queue_service_state(&self) -> &QueueServiceState

Borrow the R1 live unserved-queue state — the same instance the bridge dispatcher publishes every parked dispatch into.

Source

pub fn queue_declarations(&self) -> &QueueDeclarationSource

Borrow the R1 queue-declaration source — the engine-backed reader the bridge classifies against. Answers Unknown on a state built without an engine, which never refuses anything.

Source

pub fn unserved_queues(&self) -> Result<Vec<UnservedQueue>, ServerError>

Every queue address currently unserved, with its taxonomy reason, the policy it is held under, the live poller census behind the verdict, and the runs parked on it.

This is the server-side answer to “is anything stuck, and on what” — the question the pre-R1 seam could only be asked by reading logs.

§Errors

Returns ServerError::LockPoisoned if the state lock is poisoned.

Source

pub fn unrecoverable_runs( &self, ) -> Result<Vec<(WorkflowId, UnrecoverableRun)>, ServerError>

Every run this engine process could not make resident, with the reason it could not and when the failure was observed (#117).

This is the fleet half of the degraded-residency question. POST /workflows/describe answers it for a run an operator can already name; this answers it for the operator who cannot, which is the case that made the original defect unrecoverable in practice — the id was only ever printed in a boot log line that had scrolled away.

The set is per-process and self-clearing: an entry disappears the moment the engine observes that run resident, so an EMPTY list is the healthy answer and never a stale one.

§Errors

Returns ServerError when the state carries no engine handle, or when the registry lock is poisoned.

Source

pub fn intervention_router(&self) -> InterventionRouter

Build the NOI-6 intervention router over the connected-worker registry, the attempt-owner back-index, and the active intervention transport.

The transport is the liminal server-push (LiminalInterventionTransport) when the liminal-transport feature is compiled in — the production path that pushes a routed command out on the owning worker’s connection — and a null transport otherwise, which reports the target unreachable so every command NACKs the attempt-scoped no-op rather than silently vanishing. The router is cheap to build (it clones cloneable handles), so it is constructed per request at the endpoint rather than stored.

Source

pub fn cluster_self_node(&self) -> Option<&str>

This node’s configured cluster distribution name for the WS3 snapshot self-identity, or None on a single-node boot (the snapshot then reports the standalone self-label).

Source

pub fn engine(&self) -> Result<Arc<Engine>, ServerError>

Clone the live engine handle the completion path records terminals through.

This is the SAME Arc<Engine> the gRPC completion callback is built over (state.rs installs ServerOutboxDeliveryCallback::new(engine) on the pending tracker when outbox.enabled), so the liminal completion path re-enters worker results through the identical record_fan_out_completion seam rather than inventing a second one.

§Errors

Returns ServerError when the namespace resolver has no engine handle (a state built from parts without an engine).

Source

pub fn pending_activities(&self) -> &PendingActivities

Borrow the pending-activities tracker shared by the NIF bridge and worker stream handler.

Source

pub fn heartbeat_tracker(&self) -> &HeartbeatTracker

Borrow the heartbeat/liveness tracker shared by dispatch and worker streams.

Source

pub fn grpc_liveness_waiters(&self) -> &GrpcLivenessWaiters

Borrow the gRPC liveness answer-correlation registry (#197).

The worker stream handler delivers each LivenessAnswer into it; the liveness probe arms and awaits through the SAME handle. There is exactly one per server, for the same reason there is exactly one probe.

Source

pub fn drain_state(&self) -> &DrainState

Borrow the drain gate shared by transports and worker dispatch.

Source

pub fn metrics(&self) -> Option<&Metrics>

Borrow the prometheus metrics handle when this state was built with a store.

Source

pub fn health(&self) -> Option<&HealthState>

Borrow health probe state when this state was built with a store.

Source

pub fn activity_mock_registry(&self) -> Option<&ActivityMockRegistry>

Borrow the shared per-run activity-mock registry when the dev surface is commissioned. Returns None on a server with the dev surface dark, so the dev handlers refuse cleanly rather than mocking on a production server.

Source

pub fn outbox_store(&self) -> Option<Arc<dyn OutboxStore>>

Borrow the outbox store the dispatcher claims rows from, when the durable (haematite) backend is in use. This is the SAME leaf Arc<HaematiteStore> the engine writes through, so the dispatcher shares its single haematite::Connection rather than opening a second contending one. Returns None for the in-memory backend, which has no outbox table.

Source

pub fn namespace_store(&self) -> &Arc<dyn NamespaceStore>

Borrow the durable namespace registry shared by the control plane.

This is the SAME concrete leaf backend the engine writes events through (haematite quorum-replicated, or in-memory local-only), captured as a NamespaceStore before the decorator chain wrapped it. Always present on every boot, so the mint-on-register path (Phase 1 S5) and GET /namespaces (S7) can reach a real registry regardless of backend.

Source

pub fn worker_deployment_store(&self) -> &Arc<dyn WorkerDeploymentStore>

Borrow the durable worker-deployment store shared by the control plane.

Source

pub fn worker_supervisor(&self) -> &Arc<WorkerSupervisor>

Borrow the managed-worker supervisor.

Built over the SAME durable deployment store as Self::worker_deployment_store, so desired state written through the API is the desired state the supervisor converges on. Uncommissioned until [crate::run] installs an operator policy.

Source

pub fn namespace_minter(&self) -> NamespaceMinter

Build the shared minted-on-use hook over the durable namespace store and the configured AutoCreate policy.

This is the SAME policy logic the worker-registration seam applies (S5); the workflow-start safety net (S6) calls it after authorization so a client that starts a workflow before any worker registers still gets a durable namespace record. Cheap to build (clones an Arc + a Copy policy), so transports construct it per request rather than holding it.

Source

pub fn namespace_routing(&self) -> Option<NamespaceRouting>

The namespace-mint routing context for this boot, or None when there is nothing to route to.

Present only when ALL THREE handles exist: the distributed store (which hashes a namespace to its registry shard), the R-2 shard directory (which resolves that shard’s current owner), and the R-3 request forwarder (which dials it). Those three are populated together by build_routing_state on a [store.cluster] boot and are all None otherwise, so a partial context can never arise — but each is checked rather than assumed.

Source

pub fn outbox_wake(&self) -> Arc<Notify>

Clone the advisory outbox wake (LSUB-2) shared with the engine’s stage seam. The outbox dispatcher installs this handle so a committed fan-out row wakes its run loop in ~RTT rather than waiting for the next poll tick. The handle is always present; it is simply never pulsed when the outbox is not commissioned, so wiring it is free and behaviour is unchanged.

Source

pub fn is_clustered(&self) -> bool

Whether this server is a node in a distributed haematite cluster.

true when boot constructed the distributed backend (a [store.cluster] section was present) and is holding its inbound-write responder alive; false for every single-node / non-haematite boot.

Source

pub fn cluster_store(&self) -> Option<&Arc<HaematiteStore>>

The concrete distributed haematite store the request-routing edge consults for shard ownership (shard_for_workflow / owns_workflow_shard) and unsteered-start remint. None for every single-node / non-clustered boot, so the routing pre-step is a no-op and the default path is unchanged.

Source

pub fn shard_directory(&self) -> Option<&Arc<StaticShardDirectory>>

The request-routing shard directory (R-2) the edge consults to resolve a non-owned shard’s owner. None for single-node / non-clustered boots, so the edge falls back to the bare R-1 ownership check.

Source

pub fn request_forwarder(&self) -> Option<&Arc<dyn RequestForwarder>>

The R-3 request forwarder used to relay a non-local signal/query/cancel to the shard owner. None for single-node / non-clustered boots.

Source

pub fn spawn_heartbeat_sweeper( &self, shutdown: Receiver<bool>, ) -> JoinHandle<()>

Spawn the worker heartbeat expiry sweeper (#176): the production driver of HeartbeatTracker::fail_expired_workers, failing every worker with an in-flight task beyond the operator’s worker.heartbeat_window and deregistering it with the provable WorkerDeathReason::Timeout.

Always spawned on the server boot path — dead-worker detection is a liveness correctness property, not an opt-in feature. The cadence is derived from the heartbeat window (sweep_interval: a quarter of the window clamped to [1s, window], so the default 30s window sweeps every 7.5s); there is deliberately no separate config knob. The task exits when shutdown flips to true, exactly like the transports; the returned handle may be dropped to detach it (dropping a tokio JoinHandle never cancels the task) and is returned so tests can await clean shutdown.

Source

pub fn spawn_liminal_liveness_probe( &self, notifier: Arc<LiminalConnectionNotifier>, shutdown: Receiver<bool>, ) -> JoinHandle<()>

Spawn the liminal connection dead-man switch (the liveness probe) over notifier.

Always spawned on a boot that hosts the liminal worker listener: connection liveness is a correctness property of the transport, not an opt-in feature. Both timings derive from the operator’s worker.heartbeat_window — see LivenessProbe — so there is no separate knob. The task exits when shutdown flips to true, exactly like the heartbeat sweeper and the transports.

Source

pub fn spawn_cluster_supervisor( &self, config: SupervisorConfig, shutdown: Receiver<bool>, ) -> Result<bool, ServerError>

Spawn the SS-5b cluster supervisor: a background task that watches every declared peer’s replication liveness and, on a confirmed peer death, calls adopt_shards for that peer’s shards on THIS node’s live engine — automatic failover with no manual trigger.

Does nothing (returns Ok(()) without spawning) unless this is a distributed boot whose cluster config declared at least one peer with owned_shards. A single-node / non-clustered server therefore never runs a supervisor, so default behaviour is unchanged.

The spawned task drains on shutdown exactly like the transports.

§Errors

Returns ServerError when the engine handle cannot be resolved.

Source

pub fn shutdown(&self) -> Result<(), ServerError>

Shut down the embedded engine so in-flight durable appends can finish.

§Errors

Returns ServerError if the namespace resolver has no engine handle or the engine rejects shutdown.

Trait Implementations§

Source§

impl Clone for ServerState

Source§

fn clone(&self) -> ServerState

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> FromRef<T> for T
where T: Clone,

Source§

fn from_ref(input: &T) -> T

Converts to this type from a reference to the input type.
Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoRequest<T> for T

Source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
Source§

impl<L> LayerExt<L> for L

Source§

fn named_layer<S>(&self, service: S) -> Layered<<L as Layer<S>>::Service, S>
where L: Layer<S>,

Applies the layer to a service and wraps it in Layered.
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more