Skip to main content

WorkerJoin

Struct WorkerJoin 

Source
pub struct WorkerJoin {
Show 14 fields pub controller: Option<String>, pub ssh: Option<SshConfig>, pub token: Option<String>, pub bin: Option<String>, pub source: Option<WorkerSource>, pub libtorch: Option<String>, pub host: Option<String>, pub devices: Option<Vec<u8>>, pub persist: bool, pub data_path: Option<String>, pub data_source: Option<String>, pub gpu_ram_share: Option<f64>, pub sig_probe: Option<bool>, pub args: Vec<String>,
}
Expand description

Top-level join: block — the worker-side counterpart of controller.join:. Carries defaults for fdl join so a golden image (or a systemd unit) can bake the whole dial-in recipe into config and run bare fdl join. Every field is overridable on the command line; flags win over this block.

Fields§

§controller: Option<String>

Controller mux address, host[:port] (default port 1337). Under ssh: this is the address as seen FROM the ssh host — usually loopback, the guardrailed-sshd-on-the-controller-box case.

§ssh: Option<SshConfig>

SSH tunnel hop: fdl join brings up a local -L forward of the controller mux port through this host and dials loopback. Same shape as a worker’s ssh: sub-block (target / port / user / identity_file / options).

§token: Option<String>

Pre-shared session credential (the run’s controller.join.token), hex. Unset = open admission (the controller hands the salt out in the accept reply).

§bin: Option<String>

Training binary to run in agent role, as a path on this box: use it as given. Mutually exclusive with source:, and one of the two is required — the binary IS the protocol (it dials, joins, and spawns this host’s relay + rank children itself), so fdl join cannot default it.

This is also the escape hatch for a box whose owner wants the last word on what it compiles and runs: a declared binary wins over any source a controller would hand it.

§source: Option<WorkerSource>

Build the training binary here instead of naming one. The box fetches the tree to local disk and compiles it against its OWN libtorch, which is what makes the ABI match by construction rather than by manifest discipline.

§libtorch: Option<String>

libtorch variant to acquire before building or running: auto, cpu, cu126, cu128, rocm7.0. It lands under ~/.flodl/libtorch/ (never the project tree, which on a walk-in is often a read-only mount) and becomes this box’s active variant. Unset leaves the box on whatever it already has.

auto is the fleet value: it routes on the devices THIS box has, so one golden image serves NVIDIA and AMD instances.

§host: Option<String>

Logical host name presented in the join hello (default: this machine’s hostname).

§devices: Option<Vec<u8>>

GPU device indices to offer, one rank each (default: every device detection sees for the training build’s vendor).

§persist: bool

Keep dialing across runs: when the agent exits (run finished, controller gone, no window open yet) fdl join backs off and re-dials instead of exiting. The systemd / golden-image mode.

§data_path: Option<String>

Dataset source root on THIS box — a local path, the same role cluster.workers[].data_path plays for a fan-out host. fdl join verifies it before dialing and ships it to this host’s ranks, so the training binary needs no data flag. Unset ships nothing: the binary keeps its own default.

With data_source: set this is the MOUNTPOINT, defaulting to DEFAULT_DATA_PATH.

§data_source: Option<String>

Transport that establishes data_path: when it is not already there, <scheme>://<target>. One scheme ships today:

sshfs://[user@]host[:port]/abs/path
sshfs://[user@]host:/abs/path        # the scp spelling, same thing

A source already mounted by provisioning needs nothing here — name its path in data_path: and leave this unset. The mount goes up READ-ONLY: a rank reads the source root and never writes it, so the kernel, not a convention, enforces that.

§gpu_ram_share: Option<f64>

Integrated-GPU host-RAM share for THIS box: the fraction of MemTotal its GPU aperture claims (the library’s gpu_ram_share knob; discrete GPUs ignore it). Same role as a fan-out host’s cluster.workers[].gpu_ram_share, riding the same envelope road: the agent writes it into this host’s block at join time, where it overrides any cluster-scope default the controller stamped. Non-negative fraction of MemTotal (above 1.0 is legal where the platform under-states the aperture); unset ships nothing.

§sig_probe: Option<bool>

Probe the training binary for its model signature before each dial (default: on). The probe re-runs the binary with FLODL_INTERNAL_MODEL_SIG_PROBE set; it builds its model on CPU, prints the signature and exits, and admission can then refuse a box building a different model without costing the run. false skips it — the formation-time check remains — for a binary whose startup is too heavy to run twice per dial, or one built against a flodl that predates the probe (which would otherwise run its whole main until the probe timeout kills it).

§args: Vec<String>

Arguments for the training binary (the binary’s own training flags). A -- tail on the command line replaces this list.

Trait Implementations§

Source§

impl Clone for WorkerJoin

Source§

fn clone(&self) -> WorkerJoin

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for WorkerJoin

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for WorkerJoin

Source§

fn default() -> WorkerJoin

Returns the “default value” for a type. Read more
Source§

impl<'de> Deserialize<'de> for WorkerJoin

Source§

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>
where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DeserializeOwned for T
where T: for<'de> Deserialize<'de>,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.