pub struct PipelineCores {
pub journal_seq: Placement,
pub matching: Placement,
pub response: Placement,
pub reader: Placement,
pub event_publisher: Placement,
pub shadow: Placement,
pub repl_handler_0: Placement,
pub repl_handler_1: Placement,
pub journal_prep: Placement,
pub journal_disk: Placement,
}Expand description
Core assignments for pipeline threads, each with its wait policy.
All fields are always stored; event-publisher is only used when
--event-bind is set, and shadow only when --snapshot-interval-ms > 0.
repl-handler-0/1 are spawned on replica connect, not at startup.
A core of 0 = unpinned (OS scheduled) for any field.
The thread that accepts replica connections (repl-accept) has no
field: it accepts, reaps finished handlers and sleeps, so a core of its
own would buy nothing.
Fields§
§journal_seq: PlacementThe journal’s sequencing thread: orders, encodes and hash-chains events, and feeds the replicas and the disk thread.
matching: Placement§response: Placement§reader: Placementio_uring reader thread (TCP) or DPDK poll thread — see
ReaderThread for how the two differ.
event_publisher: Placement§shadow: Placement§repl_handler_0: PlacementReplication handler thread 0.
repl_handler_1: PlacementReplication handler thread 1.
journal_prep: PlacementThe journal segment preparer (background staging of the next segment).
Its wait policy is always yielding: the preparer blocks in file I/O and sleeps between attempts, it never polls a ring. The parser only produces a yielding preparer, and the layout check and the rendering read it as yielding whatever this field holds, so a busy-spin value set in code is ignored rather than refused. It is carried here so the layout check knows it is harmless to share a core with.
journal_disk: PlacementThe journal disk thread — the half that writes, syncs, and publishes durability.
Unlike the preparer this is a hot-path thread: it polls for
batches, and the durability cursors every ack gates on are
published from it. Place it on the same CCD as journal-seq — the
hand-off bounces a cache line between the two on every batch,
and a cross-CCD transfer costs ~100 ns of that.
Implementations§
Source§impl PipelineCores
impl PipelineCores
Sourcepub fn stage_waits(&self) -> StageWaits
pub fn stage_waits(&self) -> StageWaits
The per-stage policies the pipeline builders take, drawn from the threads that own each stage: the reader produces into the input ring, journal-seq feeds the write and replication rings, matching produces into the output ring.
Sourcepub fn place_journal_children<E: AppEvent>(&self, stage: &mut JournalStage<E>)
pub fn place_journal_children<E: AppEvent>(&self, stage: &mut JournalStage<E>)
Hand the journal stage the placements of the two helper threads its
start launches: the segment preparer and the disk thread.
journal-seq’s own placement is applied where that thread is
spawned, like every other stage’s; the helpers are launched by the
stage, so their placement has to reach it before start. One
function so the primary, the DPDK primary and the replica cannot
drift apart on which fields they forward.
Sourcepub fn all_yielding(self) -> Self
pub fn all_yielding(self) -> Self
The same layout with every thread yielding: a y on every entry,
for code that keeps a layout’s cores but must share them. Code
that only needs the node out of everyone’s way wants
unpinned instead, which spreads the threads
rather than stacking them on the layout’s cores. Not valid for a
DPDK reader, which cannot yield; see resolve.
Sourcepub fn unpinned_mandatory(&self) -> Vec<&'static str>
pub fn unpinned_mandatory(&self) -> Vec<&'static str>
The mandatory threads with no core of their own, in --cores
order: what the boot log warns about. A thread without a core runs
wherever the scheduler puts it and shares that core with whatever
else is there. On an auxiliary thread that is the documented trade
for a small box; on one of these five it is paid on every request,
so it should be a choice the operator made knowingly. A Vec only
to carry the names into one log line; this runs once, at startup.
Sourcepub fn pins_nothing(&self) -> bool
pub fn pins_nothing(&self) -> bool
Whether no thread at all has a core: the --cores none layout.
Sourcepub fn resolve(self, reader: ReaderThread) -> Result<Self, String>
pub fn resolve(self, reader: ReaderThread) -> Result<Self, String>
The layout the node actually runs: the reader entry checked
against what the transport’s reader thread can do, then the whole
thing checked with validate. Every spawn site
reads the result, so no thread waits under a policy the operator
did not ask for.
On DPDK the reader is the NIC poll thread, which busy-polls
whatever its entry says, so a y there is a contradiction and is
refused. An unpinned DPDK reader is accepted as written: it polls
flat out wherever the scheduler puts it, which is the one
placement this module cannot make honest, and the operator docs
say so.
An Err is a configuration the node refuses to start with.
Sourcepub fn validate(&self) -> Result<(), String>
pub fn validate(&self) -> Result<(), String>
Refuse a layout in which threads would starve each other: two threads pinned to the same core where either one busy-spins, or a busy-spinner with no core at all.
Checked over every thread, whether or not today’s flags spawn it
— a layout is meant to be right independent of which features are
on, and the message names the threads and the core so the fix is
obvious. Two yielding threads on one core is the supported way to
run on a small box and passes. The parser and the Placement
constructors never produce an unpinned spinner, but the fields are
public, so the check does not rely on that.
Sourcepub fn compact(num_cpus: usize) -> Result<(Self, usize), String>
pub fn compact(num_cpus: usize) -> Result<(Self, usize), String>
Compact pipeline layout that fits on num_cpus physical+logical
cores while reserving one core for an external client (bench or
reader). Used by the embedded bench so it doesn’t HT-collide with
the pipeline cores.
On the default workstation Default layout the journal (core 1),
matching (core 2), and shadow/repl_handler cores (7/8/9) are HT
siblings of each other on an 8-core (16-thread) Ryzen — a layout
designed for a 10-physical-core box. This packs everything into
the lower physical cores and leaves core 10+ free.
Returns the chosen layout and the recommended bench/client core.
Errors if num_cpus is too small to host the pipeline.
Trait Implementations§
Source§impl Clone for PipelineCores
impl Clone for PipelineCores
impl Copy for PipelineCores
Source§impl Debug for PipelineCores
impl Debug for PipelineCores
Source§impl Display for PipelineCores
Renders the layout in --cores syntax, suffixes included, so what
the node logs at boot can be pasted straight back onto a command
line. Every thread is named, unpinned ones as =0, so the log shows
the whole layout rather than only what was pinned; a layout with
nothing pinned renders as none.
impl Display for PipelineCores
Renders the layout in --cores syntax, suffixes included, so what
the node logs at boot can be pasted straight back onto a command
line. Every thread is named, unpinned ones as =0, so the log shows
the whole layout rather than only what was pinned; a layout with
nothing pinned renders as none.