Struct Transaction 

pub struct Transaction { /* private fields */ }

A transaction, alive for as long as this handle is.

Obtained from Client::start_transaction. It derefs to a Client bound to it, so every command sent through it happens inside the transaction:

let tx = client.start_transaction()?;

tx.create("table", "//tmp/out")?;
tx.write_table("//tmp/out", &rows)?;

tx.commit()?;                     // now //tmp/out exists, with its rows

Dropping it aborts it. That is what makes the ? on those two lines safe: a failure anywhere returns from the function, the handle drops on the way out, and the cluster is left as it was. Only Transaction::commit publishes anything.

Implementations

impl Transaction

pub fn id(&self) -> &str

The transaction’s ID, as the cluster named it.

Worth logging: it is what identifies the transaction in the web UI, and what Client::with_transaction needs to rejoin it from elsewhere.

pub fn client(&self) -> &Client

The client bound to this transaction.

Rarely needed — Transaction derefs to it — but a &Client is what a function taking one wants to be handed.

pub fn commit(self) -> Result<()>

Publishes everything done in the transaction.

Errors

Returns ClientError if the commit fails, which leaves the transaction aborted and nothing published: the handle is consumed either way, and a commit that did not land drops through Drop, which sends the abort. A failed commit that was left neither committed nor aborted would hold its locks until it expired.

pub fn abort(self) -> Result<()>

Discards everything done in the transaction.

The same thing dropping the handle does, for when it should read as a decision rather than as a scope ending.

Errors

Returns ClientError if the request fails. The transaction expires on its own either way, once nothing is pinging it.

pub fn ping(&self) -> Result<()>

Tells the cluster the transaction is still wanted.

The handle does this on its own; this is for a process that wants to check the transaction is still there — a ping is how the cluster reports that it is not.

Errors

Returns ClientError if the transaction has expired or was aborted.

pub fn is_lost(&self) -> bool

Whether the keep-alive has given up on this transaction.

The pinging thread stops on its own for exactly one reason: the cluster answered a ping with “no such transaction”, which is final — the transaction expired, or somebody else aborted or committed it. Without this the thread’s exit is invisible, and a handle that has quietly stopped pinging looks exactly like a healthy one until the next command fails.

So this is for a holder that keeps a transaction across something long: a false answer means only that no ping has been answered that way yet, which is the strongest thing a handle can say without asking, and Transaction::ping is how to ask.

False is not “something is pinging”. Two other states read false with nothing keeping the transaction alive:

• the thread never started, because the spawn failed. Nothing has been lost and nothing is pinging either, so the transaction runs on the cluster’s clock from whenever it was last pinged.
• the thread panicked. Nothing on the ping path panics as it stands — a poisoned lock is recovered rather than unwrapped — so this is about a future edit to that path rather than about the code today.

Neither is visible from the handle, and a ping does not expose them either: it answers for the transaction, not for the thread, so it goes on succeeding until the transaction actually expires. What they have in common is the remedy — ping, or attach afresh.

This is also &self while Transaction::detach consumes the handle, so there is nothing left to ask once a transaction has been detached. From there the only probe is Client::ping_transaction on the id.

pub fn detach(self) -> String

Stops keeping the transaction alive and leaves it running.

The deliberate exception to what Drop promises: the transaction survives the handle. Nothing is committed, aborted or otherwise decided — to the cluster a detached transaction looks exactly like a held one — so from here it lives on the cluster’s terms: it expires its timeout after its last ping, 30 seconds by default, unless something else keeps it alive. That something is the point: hand the returned id to another process, which re-holds it with Client::attach_transaction or finishes it outright with Client::commit_transaction or Client::abort_transaction.

The keep-alive is asked to stop and then waited for, for up to five seconds. Inside that bound nothing is left in flight, and the caller can kill the process the moment this returns without a stray request behind it. What the wait is for, and where it gives up:

• The keep-alive may get one last ping away — it can be past its own stop check and about to send when detach raises the flag — so the transaction’s clock may restart once more, at up to one ping after this was called. That ping is what the wait is for.
• Past five seconds the ping is left in flight and this returns anyway, rather than hold the caller’s thread. A ping has a request budget of its own — min(interval / 2, 120 s), on an interval of a third of the transaction’s timeout — and five seconds covers that whole budget while the timeout is under 30 seconds, equalling it at the 30 s default. So at or below the default the wait genuinely ends in the thread’s exit. Above the default it need not: an hour-long launcher transaction pings on a two-minute budget, and a ping stalled on a proxy that has stopped answering outlasts the wait, reaches the master after detach returned, and restarts the expiry clock there — the transaction lives a full timeout from wherever that ping landed rather than from this call. Nothing is leaked: the thread re-reads the stop flag as soon as its ping ends, so at most one ping is outstanding and it exits inside that same budget. But it is alive and unreaped past the detach, and a caller whose timeout is above the default cannot treat this call as the transaction’s last ping.

What C++ spells ITransaction::Detach(). It is also the honest way to let a transaction outlive its handle: mem::forget on a Transaction leaks the keep-alive thread, which goes on pinging for the life of the process and holds the transaction and its locks open indefinitely.

Methods from Deref<Target = Client>

pub fn transaction_id(&self) -> Option<&str>

The transaction this client is bound to, if any.

pub fn traceparent(&self) -> Option<&str>

The traceparent header this client sends, if it was given one.

pub fn tracestate(&self) -> Option<&str>

The tracestate header this client sends, if the context it joined carried one. See TraceContext::with_tracestate.

pub fn start_transaction(&self) -> Result<Transaction>

Starts a transaction, and keeps it alive while the handle lives.

Everything sent through the returned Transaction is invisible to everything else until it commits, and is discarded if it does not:

let tx = client.start_transaction()?;

tx.create("table", "//tmp/out")?;   // no one else can see it yet
tx.write_table("//tmp/out", &rows)?;

tx.commit()?;                       // and now everyone can

The transaction lasts 30 seconds without a ping — the cluster’s own default — and the handle pings it every ten, so an operation that runs for an hour is fine. Client::start_transaction_with changes the timeout.

Errors

Returns ClientError if the transaction cannot be started.

pub fn start_transaction_with(&self, timeout: Duration) -> Result<Transaction>

Starts a transaction that expires timeout after its last ping.

The handle pings three times per timeout, so this is about what happens when the handle is gone: how long the transaction holds its locks after the process holding it dies without aborting. Shorter frees them sooner; longer survives a longer pause.

Errors

Returns ClientError if the transaction cannot be started.

pub fn attach_transaction(&self, id: &str) -> Result<Transaction>

Attaches to a transaction something else started, and keeps it alive.

The receiving half of Transaction::detach: one process starts a transaction and detaches, hands the id over, and this turns the id back into a real Transaction — a bound client, a pinging thread, and commit/abort/ping that work. Two things differ from a handle the same process started, and both follow from not being the owner:

• Dropping it detaches rather than aborts — the pings stop and nothing is sent. The C++ client’s destructor draws the same line, and for the same reason: an attacher’s ? must not destroy work the process that started the transaction is still counting on. An explicit Transaction::abort still aborts; only the drop differs.
• The ping interval is read, not chosen. Pinging needs the transaction’s timeout and the id alone does not carry it, so this asks the cluster for #<id>/@timeout — one round trip, which is also what makes attaching to a transaction that is gone fail here, rather than on the first command sent through the handle.

It pings before it returns, one more round trip. @timeout is the configured lifetime and says nothing about how much of it is left: the id carries no hint of when its last holder pinged, so a handoff that took longer than two thirds of the timeout would otherwise hand back a handle whose first ping is already too late. That ping restarts the cluster’s clock at the attach, and doubles as the liveness probe this call reports on.

So this is two retryable round trips, both on this client and so under its retry policy — five attempts of two minutes by default, backoff between — where the keep-alive’s own pings run one attempt on a budget of half the ping interval. A ping the caller is waiting on should not fail over one dropped packet; a keep-alive ping is retried by being sent again next interval.

Nothing stops two attaches to the same id. Each is a real handle with a thread of its own, and they simply ping the same transaction twice as often; whichever commits or aborts first decides it, and the other’s next command fails with No such transaction. There is no registry, on purpose — a second process attaching is the whole point, and this process is not in a position to know about it.

The handle always pings. One that did not would be Client::with_transaction — the plain binding, which already exists — plus Client::ping_transaction, Client::commit_transaction and Client::abort_transaction, which take the bare id; reach for those where a thread per transaction is not wanted. (The Go SDK spells that choice AttachTx(id, &AttachTxOptions{AutoPingable: false}).)

let tx = client.attach_transaction(&id_from_elsewhere)?;

tx.create("table", "//tmp/out")?;   // inside the shared transaction
tx.commit()?;                       // and now published, by this process

Errors

Returns ClientError if the transaction does not exist or the timeout cannot be read. The error names the id and the operation itself, because the cluster’s own answer does not always do either. Both spellings were observed on a local cluster: an expired id earns Error resolving path #<id>/@timeout around No such object <id> — object, not transaction, since the id is addressed as one — while an id that never named anything is refused as Unknown cell tag 0, with no id in it at all. A transaction that expires between the two round trips fails the same way, on the ping: No such transaction.

pub fn ping_transaction(&self, id: &str) -> Result<()>

Tells the cluster a transaction is still wanted, by bare id.

A held Transaction does this on its own thread; this is for a process that has nothing but the id — between a Transaction::detach in one process and the commit in another, somebody must say the transaction is still wanted, or it expires its timeout after its last ping (30 seconds by default; verified on a local cluster with a two-second timeout left alone for four). A ping is also the cheapest liveness probe: the cluster answers one for a transaction that is gone with No such transaction.

Errors

Returns ClientError if the transaction has expired, was aborted, or never existed.

pub fn commit_transaction(&self, id: &str) -> Result<()>

Publishes everything done in a transaction, by bare id.

What lets a process finish a transaction it did not start — the other end of a Transaction::detach, without the round trip and the ping thread of Client::attach_transaction.

Sent under a mutation ID, because a commit is not idempotent: the second commit of the same transaction is refused with No such transaction, which reads like the first one failed. The mutation ID makes a retried commit the same commit rather than a second one.

Errors

Returns ClientError if the commit fails — including No such transaction for one that expired, was aborted, or was already committed.

pub fn abort_transaction(&self, id: &str) -> Result<()>

Discards everything done in a transaction, by bare id.

Forgiving, unlike Client::abort_operation: aborting a transaction that already committed, aborted or expired — or one that never existed — answers {}, verified on a local cluster. So this is safe to send on any cleanup path, and it is retried freely on the same grounds.

Errors

Returns ClientError if the request fails. The transaction expires on its own either way, once nothing is pinging it.

pub fn heavy_proxy(&self) -> Result<Option<String>>

Asks the cluster for the least-loaded heavy proxy, if it has one.

The client already does this for itself. Heavy commands — table and file data, in either direction — resolve a heavy proxy on their own and go there; see the module documentation for when, and for how long the answer is kept. So this is no longer the way to make an upload work: it is the way to see the address, or to hand it to something that is not this client — a second Client, another process, a curl.

It asks every time and shares nothing with what the client resolved for itself, so calling it neither costs nor changes anything the next command does. It also reports the name as the cluster gave it, before the checks automatic routing puts it through — which is what makes it the way to see why a host was declined. A name here that the uploads are not using is the symptom Client::with_heavy_proxies_anywhere exists for.

It shares the lookup’s budget, though: one attempt bounded by Client::with_hosts_timeout — 800 ms unless that says otherwise — rather than the client’s retry policy and request timeout. The budget belongs to the question, not to whoever asked it.

Errors

Returns ClientError if the request fails, or if /hosts does not answer with the documented list of host names. Ok(None) means the cluster answered and named no heavy proxy — which a failure must not be allowed to look like, since the caller’s next move is to stop looking.

pub fn exists(&self, path: &str) -> Result<bool>

Whether a Cypress node exists.

Errors

Returns ClientError if the request fails.

pub fn create(&self, node_type: &str, path: &str) -> Result<()>

Creates a Cypress node, e.g. table, file or map_node.

Creates missing parents and succeeds if the node already exists.

Errors

Returns ClientError if the request fails.

pub fn create_table(&self, path: &str, schema: &TableSchema) -> Result<()>

Creates a table with a schema.

A schematised table is checked on every write, stores its columns in their own types, and can be sorted and merged; an unschematised one takes anything and finds out later.

let schema = TableSchema::new([
    Column::new("host", ColumnType::Utf8).required().key(),
    Column::new("size", ColumnType::Int64).required(),
]);
client.create_table("//tmp/visits", &schema)?;

Unlike Client::create, this fails if the path already exists. That is deliberate: the cluster ignores the attributes of a create it skips, so an ignore_existing version of this would quietly leave the old table with the old schema and report success. Changing the schema of a table that exists is alter_table’s job.

Errors

Returns ClientError::Config if the schema is one the cluster would refuse, or ClientError if the request fails.

pub fn alter_table(&self, path: &str, schema: &TableSchema) -> Result<()>

Changes the schema of a table that already exists.

The other half of Client::create_table: a table outlives the program that made it, and the rows it holds gain columns.

let wider = TableSchema::new([
    Column::new("host", ColumnType::Utf8).required().key(),
    Column::new("size", ColumnType::Int64).required(),
    Column::new("referrer", ColumnType::Utf8), // new, and optional
]);
client.alter_table("//tmp/visits", &wider)?;

A table with rows in it accepts only changes that ask less of the rows already written. Watched on a cluster, on a table holding two rows — and each refusal says which column and why:

Change
add an optional column, anywhere in the order	allowed
make a required column optional	allowed
strict → non-strict	allowed
add a required column	Cannot insert a new required column "must" into a non-empty table
remove a column	Cannot remove column "size" from a strict schema
change a column’s type	Type … is modified in non backward compatible manner
rename a column	read as a removal, and refused as one
make the table sorted	Cannot change schema from unsorted to sorted
non-strict → strict	Changing "strict" from "false" to "true" is not allowed

Two consequences worth knowing before either becomes permanent:

• An empty table accepts all of it — dropping columns, changing types, becoming sorted. So a schema change tried out on an empty table proves nothing about the same change on a full one.
• A non-strict schema can never gain a named column: Cannot insert a new column "note" into non-strict schema. Relaxing strict is a one-way door out of schema evolution.

Unlike create, the schema here is a top-level parameter rather than an attribute — the two commands are exact opposites on this, and create silently ignores the spelling alter_table requires.

Errors

Returns ClientError::Config if the schema is one the cluster would refuse outright, or ClientError if the change is rejected as incompatible.

pub fn table_schema(&self, path: &str) -> Result<YsonValue>

The schema of a table, as the cluster stores it.

Returns the raw YSON: the cluster answers with more than it was given — every column carries required, type and type_v3 whichever was written, and the keys come back in alphabetical order.

Errors

Returns ClientError if the request fails.

pub fn remove(&self, path: &str) -> Result<()>

Removes a Cypress node.

The node must exist, and a map node must be empty — the cluster’s own defaults, and the safe ones: a mistyped path fails instead of deleting whatever it happened to name. Client::remove_tree is the deliberate spelling for a subtree.

Errors

Returns ClientError if the node does not exist, is a non-empty map node, or the request fails.

pub fn remove_tree(&self, path: &str) -> Result<()>

Removes a Cypress node and everything under it. Succeeds if it is already absent.

This is recursive plus force: the spelling for “make this path not exist”, whatever is there now — which is also why it deserves a moment of care with the argument.

Errors

Returns ClientError if the request fails.

pub fn list(&self, path: &str) -> Result<Vec<String>>

The names of a node’s children.

Not sorted. The order is the cluster’s own and has no meaning; a listing of three dated tables came back as the second, the third and then the first. Sort it if the order matters.

A path that is not a map node is an error rather than an empty list — "List" method is not supported — and so is a path that does not exist.

Errors

Returns ClientError if the request fails, or if the cluster marks the answer incomplete: a listing that is silently short is worse than no listing.

pub fn copy(&self, source: &str, destination: &str) -> Result<()>

Copies a node, creating missing parents.

Fails if destination exists; Client::copy_replacing is the one that overwrites.

Errors

Returns ClientError if the request fails.

pub fn copy_replacing(&self, source: &str, destination: &str) -> Result<()>

Copies a node over whatever is at destination.

Errors

Returns ClientError if the request fails.

pub fn move_node(&self, source: &str, destination: &str) -> Result<()>

Moves a node, creating missing parents.

Fails if destination exists; Client::move_replacing is the one that overwrites, and the pair is how a result is published: write a staging table, then move it over the live one.

Named move_node because move is a Rust keyword, and client.r#move at every call site would be a worse tax than the four extra characters.

Errors

Returns ClientError if the request fails.

pub fn move_replacing(&self, source: &str, destination: &str) -> Result<()>

Moves a node over whatever is at destination.

Errors

Returns ClientError if the request fails.

pub fn link(&self, target: &str, link_path: &str) -> Result<()>

Creates a link at link_path pointing at target.

A link resolves to its target, so //tmp/latest/@row_count reads the target’s row count. To ask about the link itself, put & after its path: //tmp/latest&/@target_path. Without the & the question goes through to the target and is answered as if the link were not there.

Fails if link_path exists; Client::link_replacing is what points an existing link somewhere else.

Errors

Returns ClientError if the request fails.

pub fn link_replacing(&self, target: &str, link_path: &str) -> Result<()>

Points a link at target, replacing whatever is at link_path.

The //tmp/thing/latest pattern: publish under a dated name, then move the link. Readers that follow the link see the old version until this call and the new one after it, and never a half-written table.

Errors

Returns ClientError if the request fails.

pub fn lock(&self, path: &str, mode: LockMode) -> Result<Lock>

Takes a lock, or fails because somebody else holds one.

Only inside a transaction: a lock lives as long as the transaction that took it, and there is nothing else for it to belong to. A client that is not in one is told so here rather than by the cluster.

The failure is worth reading — it names the transaction that won:

Cannot take "exclusive" lock for node //tmp/live since "exclusive" lock
is taken by concurrent transaction 4-dac2-10001-eb1b

Client::lock_waiting queues for it instead of failing.

Errors

Returns ClientError::Config if this client is not in a transaction, or ClientError if the lock is refused.

pub fn lock_waiting( &self, path: &str, mode: LockMode, wait_for: Duration, ) -> Result<Lock>

Queues for a lock, and waits until it is held.

A waitable lock is granted later, or never — the cluster answers immediately with a lock that is pending, and it becomes acquired when the transactions ahead of it end. Returning that lock as though it were held is the mistake this command exists to make impossible: this polls until the cluster says acquired, and gives up after wait_for.

The deadline is not a nicety. A request can queue for something that will never happen and the cluster will not say so: a transaction that already holds a snapshot lock on the node is refused an exclusive one outright, but the waitable version of the same request is queued behind a lock only that transaction’s own end will release.

Errors

Returns ClientError::Config if this client is not in a transaction or the wait ran out, or ClientError if a request fails. A lock that is still queued when the wait runs out stays queued until the transaction ends.

pub fn get(&self, path: &str) -> Result<YsonValue>

Reads a node attribute, such as @row_count.

Errors

Returns ClientError if the request fails.

pub fn row_count(&self, path: &str) -> Result<i64>

Number of rows in a table.

Errors

Returns ClientError if the request fails or the attribute is absent.

pub fn execute_batch( &self, batch: &BatchRequest, ) -> Result<Vec<Result<YsonValue>>>

Executes every part of a BatchRequest in one round trip, and answers with a Result per part.

The parts fail individually — that is the entire point of the shape. One part hitting a node that already exists does not cost the other eleven their tables, and collapsing the answers into one Result would lose exactly the thing batching makes harder to see. The outer Result is for the envelope alone: the request that could not be sent, the response that could not be read.

let mut batch = BatchRequest::new();
batch
    .create("map_node", "//tmp/pipeline")
    .create("table", "//tmp/pipeline/clicks")
    .exists("//tmp/elsewhere");

for part in client.execute_batch(&batch)? {
    match part {
        // The envelope is keyed by what each command returns —
        // `{node_id=…}` for a create, `{value=…}` for an exists.
        Ok(answer) => println!("{answer:?}"),
        Err(error) => eprintln!("{error}"),
    }
}

Each Ok carries the part’s own answer exactly as that command would have answered alone — {node_id=…}, {value=…}, {} for a set — and each Err is a ClientError::Cluster named after the part’s command, flattened outer-plus-innermost like every other cluster error here. Results come back in the order the parts went in; watched on a local cluster, where a batch of create·set·get·remove answered [error 501, ok, ok, error 500] in exactly that order. An answer with the wrong number of results, or a part result shaped like nothing this client knows, fails the whole call as ClientError::Decode rather than being read as somebody’s success.

The wire

The command is execute_batch — REGISTER_ALL(TExecuteBatchCommand, "execute_batch", Null, Structured, true, false) in the cluster’s own registry: volatile and light, so a POST. The parts travel as requests=[{command=…; parameters={…}; input=…}] and the answer is the v4 envelope {results=[{output=…}|{error=…}]} (command reference; TExecuteBatchCommand in etc_commands.cpp; both shapes confirmed against a local cluster).

The parameters go in the request body, not the X-YT-Parameters header that carries every other command’s. A batch’s parameters are the batched commands, and a header has a size nobody promises; the C++ client makes the same choice for this same command (THttpRawBatchRequest::ExecuteBatch sends the parameter node as the POST body), and the proxy reads body parameters for any POST and merges them with the header’s (TContext::CaptureParameters in context.cpp — query string, then header, then body). Measured here: requests in the body and mutation_id in the header land as one parameter set.

Retries, and what makes them safe

A batch of the typed parts retries like any light command, and a mutating one retries under a mutation id — because the cluster spreads that id over the parts. The driver takes the batch’s own id and hands part k the id plus k (Options.GetOrGenerateMutationId() then NRpc::GenerateNextBatchMutationId per part in TExecuteBatchCommand::DoExecute; the increment is ++id.Parts32[0], yt/yt/core/rpc/helpers.cpp), stamping it and the batch’s retry flag into every volatile part. A replay of the whole batch therefore replays every part under its original id, and the master’s mutation cache answers each with its first response. Measured on a local cluster: a two-BatchRequest::create_table batch sent under an explicit id, then sent again with retry=%true, answered the same two node ids both times — where the same batch under a fresh id got two 501 already exists.

The measurement uses create_table and not BatchRequest::create on purpose, and repeating it with create proves nothing: create sends ignore_existing, so a second send answers with the old node’s id whether or not the cluster recognised a replay. Measured that way too — create under a fresh id returned the same two ids as the first send, with no mutation cache involved at all. create_table omits ignore_existing, so its second send fails unless it was deduplicated, which is what makes the identical ids mean something.

That safety is the master’s, which is why the default is per-part kind: parts this crate models are Cypress commands the master’s cache covers, so their batches go out Repeatable::WithMutationId (or Repeatable::Freely when every part is a read, since such a batch mutates nothing). A BatchRequest::raw part may name a command the cache does not cover — the scheduler commands are the measured example, where a replayed id turns a success into No such operation — so a batch carrying one is sent once, exactly as Client::raw_command is.

Transactions

A client bound to a transaction puts the parts in it — each part is stamped with transaction_id, not the envelope. The envelope has no transaction to be in, and the distinction is measurable: an outer transaction_id was dropped in silence by a local cluster, the part’s create landing outside the transaction and surviving its abort. A part that already names a transaction keeps its own, and a part whose command takes none is left alone, both as the transport itself would have it.

A big batch is several requests, and a failed one leaves a prefix

More parts than BatchRequest::with_max_part_size allows are split into consecutive execute_batch requests — the C++ client’s BatchPartMaxSize behaviour, defaults included — with the results stitched back in part order and a mutation id per request. There is no rollback across them: when a later request fails wholesale, the earlier ones have already run and their parts have taken effect, the same way the C++ client’s ExecuteBatch throws with the earlier requests applied.

What this method does not do is throw that prefix away. A split batch that stops part of the way through fails with ClientError::BatchInterrupted, which carries every answer already received, in part order, beside the failure that stopped it — so a caller can see which parts landed and pick up from answered.len(). Re-running the same BatchRequest is not how to recover: a second execution mints fresh mutation ids, so the parts that already applied are applied again rather than deduplicated. Keep a batch inside one request’s worth if that matters, or give the sequence a transaction.

answered is what came back, which is not the same as what was applied, and the difference is the whole failed request. A request refused while executing has no per-part results and has nonetheless run every one of its parts — the driver collects the sub-requests into callbacks, runs them all through CancelableRunWithBoundedConcurrency, and then throws away the entire result list at .ValueOrThrow() the moment one entry is a throw. Dispatch is never aborted, so this is not a race and there is no way to arrange the parts to limit it: measured on a local cluster, a create beside a part naming an unknown command created its node with the bad part first and last, two creates around one both landed, and at concurrency=1 eight creates followed by the bad part all eight landed — every time answered Unknown command … with no results at all.

The bound worth knowing is the other one: a request refused while its parameters are being read runs nothing. Validation failed at /concurrency, Error loading parameter /requests and Missing required parameter /requests all left a create in the same request with no node behind it. Parse-time failure means none of it ran; execution-time failure means all of it did.

So the parts before answered.len() are settled, and the request that failed is unknown territory — not because some of it might have run, but because all of it did and none of it said what happened. That is what a transaction is for.

A redirect this batch cannot follow

The parts travel in the body, so this is the crate’s first light command with bytes in one — and the redirect rule reads a body as data a redirect must not hand to another origin (RedirectRefusal::Payload). A cross-origin 3xx on a batch is therefore refused where the same creates sent one at a time are bodiless POSTs the rule deliberately lets through. It is narrow — a client with a token is refused a cross-origin hop anyway, by the credentials rule — but a tokenless client behind a balancer that canonicalises to another origin finds batching breaks what individual calls did. Address the origin the balancer canonicalises to, and the hop never happens.

Errors

Returns ClientError::Config for an empty batch — the cluster would answer {results=[]} and this crate does not report a no-op as work done — ClientError::BatchInterrupted when a split batch stops after some of its requests have applied, and otherwise ClientError as any command fails. Per-part failures are not errors of this method: they are the Err halves of the vector.

pub fn execute_batch_with( &self, batch: &BatchRequest, mutation_id: Option<&MutationId>, ) -> Result<Vec<Result<YsonValue>>>

As Client::execute_batch, with a caller-supplied MutationId.

The guarantee is the one Client::raw_command_with describes and the one a single process cannot give itself: persist the id, and a batch replayed after a crash is deduplicated against the send that already happened instead of applying every part a second time. Measured on a local cluster through this method: a batch of two BatchRequest::create_table parts sent under an explicit id, then sent again under id.as_retry(), answered the same two node ids both times — where the same batch under a fresh id got two 501 already exists.

Reach for create_table and not BatchRequest::create when checking this by hand. create sends ignore_existing, which makes a second send answer with the old node’s id on its own: measured, a two-create batch under a fresh id returned ids identical to the first send’s, which looks exactly like a deduplicated replay and is not one. create_table sends no ignore_existing, so identical ids there can only be the mutation cache.

That works because the cluster spreads the id over the parts rather than deduplicating the envelope: the driver hands part k the batch’s id plus k, so a replay replays each part under the id its first send used. It is also why an id covers one request and not a split batch — see the refusal below.

let id = MutationId::new();
// …persist `id.as_str()` here, before sending…
let made = match client.execute_batch_with(&batch, Some(&id)) {
    Ok(made) => made,
    // After a crash, the same id marked as a replay: the cluster
    // answers with what the first send did, whether or not it landed.
    Err(_) => client.execute_batch_with(&batch, Some(&id.as_retry()))?,
};

An id is stamped whatever the batch’s own retry class works out to, including on an all-read batch that would otherwise carry none — the two answer different questions, as Client::raw_command_with spells out. It does not make a send-once batch retriable in-process: a batch holding an unclassified BatchRequest::raw part is still sent once.

Errors

As Client::execute_batch, and additionally ClientError::Config when an id is given for a batch that would be split into more than one request. One id cannot cover several: the driver derives each part’s id by incrementing the batch’s, so a second request under anything derived from the same id would collide with the first request’s parts and be answered with their results. Raise BatchRequest::with_max_part_size until the batch fits one request, or send it without an id.

pub fn upload_worker(&self, local: impl AsRef<Path>, remote: &str) -> Result<()>

Uploads a local file to Cypress, marking it executable.

This is what makes a worker runnable on a node: without the executable attribute YTsaurus copies the binary but refuses to exec it, and the job fails with a permission error that does not mention the attribute.

Errors

Returns ClientError if the file cannot be read or the upload fails.

pub fn upload_current_exe(&self, remote: &str) -> Result<()>

Uploads the running executable to Cypress, marked executable.

This is the one-binary pattern: the same program launches the operation and runs as its job, telling the two apart with ytsaurus_job::is_inside_job. The binary on the cluster is then by construction the one you just built — the whole “I uploaded a stale worker” class of bug disappears.

The running executable has to be something a node can exec, so its ELF header is checked before the upload: Linux, x86-64, statically linked. Launching from macOS, or from a Linux host where the launcher is dynamically linked, it is not — this returns ClientError::NotAWorker naming the reason, instead of uploading a binary that fails on the node minutes later. Build the worker with scripts/build-worker.sh and upload it with Client::upload_worker in that case.

Errors

Returns ClientError::NotAWorker if the running executable cannot run on a node, or ClientError if the upload fails.

pub fn upload_worker_cached( &self, local: impl AsRef<Path>, ) -> Result<CachedFile>

Uploads a worker, or finds it already on the cluster.

Keyed by the file’s MD5, so an unchanged binary is uploaded once and every later launch reuses it. That is the difference between a dev loop that re-sends tens of megabytes on every run and one that does not.

The cached node is named after the hash, so the returned CachedFile::name is the name to give it in the sandbox — see MapSpec::with_local_file_named:

let worker = client.upload_worker_cached("target/.../my_job")?;
let spec = MapSpec::new("./my_job", ["//tmp/in"], ["//tmp/out"])
    .with_local_file_named(&worker.path, &worker.name);

The cache is shared: Client::with_file_cache defaults to the path the Python wrapper uses, so an installation that already expires old entries there expires these too.

A cache you may not write to

On an installation where that shared path is maintained by its operators, an ordinary user may read it and nothing more — and the cluster answers a write with Access denied. That is a degraded cache, not a failed upload: the worker goes up outside the cache instead, to a path of its own under //tmp, and the launch proceeds.

It is warned about rather than passed over, on stderr — as a WARN event where the tracing feature is on — because the state is permanent until someone acts on it and invisible otherwise: every launch re-sends the whole binary, and every launch leaves a node behind that no cache expiry will collect. The warning names Client::with_file_cache, which is the one line that puts a cache back.

Only the cluster’s refusal of the cache is treated this way — creating the cache directory, creating the staging node inside it, and the handover to put_file_to_cache. Any other failure, including an Access denied on anything else, is returned.

CachedFile::cached is which of the two happened, and it is the field to read before doing anything to CachedFile::path: on the fallback path that node is this launch’s own and nobody else’s, while on the ordinary path it is the installation’s shared cache entry.

Errors

Returns ClientError if the file cannot be read or the upload fails.

pub fn file_from_cache(&self, md5: &str) -> Result<Option<String>>

Looks up a file in the cluster’s file cache by its MD5.

None means nothing is cached under that hash — including when the cache directory does not exist yet, which is what Client::upload_worker_cached creates on its way past, on a cluster that lets it.

A lookup and nothing more: it sends no mutation, so it works against a cache the caller may only read.

Errors

Returns ClientError if the request fails.

pub fn put_file_to_cache(&self, path: &str, md5: &str) -> Result<String>

Hands a file already written to Cypress to the file cache.

The cluster verifies that the node’s MD5 is the one given, which is why it must have been written with compute_md5. Returns the path the file now lives at.

Errors

Returns ClientError if the request fails.

pub fn write_file(&self, path: &str, contents: &[u8]) -> Result<()>

Writes raw bytes to a Cypress file, replacing its contents.

Errors

Returns ClientError if the request fails.

pub fn read_file(&self, path: &str) -> Result<Vec<u8>>

Reads a whole Cypress file into memory.

The mirror of Client::write_file, and the buffered half of the pair: for a worker binary fetched back, a config a launcher inspects — results, not bulk data. For a file that does not fit, Client::read_file_streaming moves the same bytes without holding them.

The whole file is held in memory, and there is a ceiling: 512 MiB. That is the transport’s cap on any buffered response, counted in the bytes that land in the Vec — and a file past it is refused rather than truncated, with a ClientError::ResponseTooLarge that names the number and names the streaming half. A file of exactly the ceiling is not past it. A worker binary is comfortably under; a dataset someone stored as a file may not be, and that is exactly the case the pair comes in two halves for.

512 MiB held is not 512 MiB of process. The buffer grows by doubling and copies as it grows, so both halves are resident for the length of a copy — up to about 1.5× the cap where the allocator cannot extend in place. Measured in a release build: a read that hands back 536 870 911 bytes peaks at 544 178 176 of resident set, and a 600 MiB read refused by the cap peaks at 611 385 344. Size for that, not for the ceiling.

The cap counts decoded bytes because the compressed ones are not the same quantity and are not close to it: this client asks for gzip, and measured against a cluster, a 600 MiB file of zeros crosses the wire in 611 522 bytes. A cap on what arrives would have let all 600 MiB into memory — which is what it did until this was fixed.

path is a plain node path — //tmp/worker. Not a rich one, and the reason is worth spelling out, because a rich path here does not fail so much as quietly do nothing. Measured on a cluster, on a file of 1000 bytes:

• <lower_limit={offset=0};upper_limit={offset=10}>//tmp/f reads back all 1000 bytes and passes the size check. A file is sliced by the command’s own offset and length parameters, not by limits on the path, so limits written there are accepted and ignored — and the caller who thought they had asked for ten bytes is told nothing. <append=%false>//tmp/f is the same story with a harmless attribute.
• //tmp/f[#0:#10] also reads back all 1000 bytes, and then fails: the size check builds {path}/@uncompressed_data_size out of this string textually, and //tmp/f[#0:#10]/@uncompressed_data_size is not a path the cluster will parse — Error reading parameter /path: Unexpected token "/" of type "slash". A whole file downloaded and then refused over a range that was never going to be honoured.

So: a plain path. Selection on reads is #12, and belongs in parameters this method would have to grow, not smuggled in through this argument.

The body’s length is checked against the size Cypress records for the node. That is not pedantry — the proxy reports a mid-stream failure in a trailer this client cannot see (see TableReader for the trailer gap), and a file’s bytes carry no framing of their own: where a truncated table leaves a record that does not parse, a truncated file just ends, looking exactly like a shorter file. So after the read, one light get fetches the node’s @uncompressed_data_size — the byte count of the content, whatever compression the node’s own codec applies beneath it — and a body of any other length is an error rather than a file.

The two requests are not atomic, and the race runs both ways. A writer replacing the file between them can fail the check for a body that was complete when it was sent — the ordinary hazard of reading what someone else is rewriting, surfaced as an error rather than as a mix of the two versions. The converse is rarer and quieter: a body genuinely cut short at N bytes, racing a replacement whose own @uncompressed_data_size is exactly N, passes the check, and a truncated read of the old version is returned as a whole file. That one cannot be closed from here — the only in-band verdict on a cut stream is the proxy’s trailer, which ureq 3.3 does not read, so there is no header to prefer over the second request. A reader who needs a file pinned while others replace it takes a LockMode::Snapshot lock in a transaction, which is exactly what that mode is for, and closes both directions at once.

Verified against a local cluster: a 4 MB Client::write_file of non-UTF-8 bytes comes back byte-for-byte through both halves of the pair, an empty file reads back empty, and a node carrying compression_codec=zlib_6 — 1 000 000 logical bytes, 4 214 on disk — reads back its logical bytes with the check passing, which is the case that would break if the attribute were the on-disk size. And a 600 MiB file of zeros — 611 522 bytes on the wire — is refused rather than held, while read_file_streaming moves all 629 145 600 of it.

Errors

Returns ClientError if the request fails, if the response is larger than the 512 MiB this holds in memory — a ClientError::ResponseTooLarge, which is never retried and never blamed on the proxy that served it — if the node’s size cannot be read — the check refuses loudly rather than quietly not happening — or if the body’s length is not the size the cluster records. A missing path fails the read itself, before the size is ever asked for: code 1, Error getting basic attributes of user objects, with the resolve error nested inside — a category outside and the reason within, as a missing table is reported too.

pub fn read_file_streaming(&self, path: &str) -> Result<FileReader>

Reads a file as a stream, without holding it.

The same bytes Client::read_file returns, arriving as they come off the connection — and a file is exactly the thing that might not fit in memory, which is why Client::write_file’s mirror comes in two halves. What comes out is a plain Read:

let mut file = client.read_file_streaming("//tmp/worker")?;
std::io::copy(&mut file, &mut std::fs::File::create("worker")?)?;

Client::read_file checks the body against the size the cluster records; this cannot, because the point is not to have the whole thing — and unlike a table, whose truncation leaves a record that does not parse, a file cut short by a mid-stream failure simply ends. A caller who needs certainty compares the reader’s bytes_read against the node’s @uncompressed_data_size — see FileReader for why that gap exists.

Errors

Returns ClientError if the request fails. Failures during the read arrive from the reader, not from here.

pub fn set_attribute( &self, path: &str, name: &str, value: YsonValue, ) -> Result<()>

Sets a node attribute.

Errors

Returns ClientError if the request fails.

pub fn write_table(&self, path: impl Into<TablePath>, rows: &[u8]) -> Result<()>

Writes rows to a table, replacing its contents.

rows must be a binary YSON list fragment — exactly what a ytsaurus-job worker writes.

A path carrying a read selection — TablePath::columns, TablePath::range, or rich YPath syntax spelled into the path string — is refused locally, before anything is sent. The cluster ignores those on a write and replaces the whole table with a 200 (measured: write_table_rows("//tmp/t[#0:#2]", rows) replaced everything and reported success), and this refusal is what keeps that silent loss unwritable. See TablePath.

Errors

Returns ClientError::Config if the path carries a read selection, or ClientError if the request fails.

pub fn write_table_with_format( &self, path: impl Into<TablePath>, rows: &[u8], format: &DataFormat, ) -> Result<()>

Writes rows to a table using a shared DataFormat, replacing its contents.

YSON data is a list fragment in the selected representation. Skiff data is a complete schema-described stream; direct table I/O requires exactly one schema with named non-system fields.

Errors

Returns ClientError if the format is unsupported, the data is not a complete Skiff stream, or the request fails.

pub fn write_skiff_table( &self, path: impl Into<TablePath>, rows: &[u8], format: &SkiffFormat, ) -> Result<()>

Writes a complete Skiff stream to one table, replacing its contents.

format must have exactly one table schema. Its named fields are sent as the rich-path columns projection, matching the Go SDK; this is how the proxy maps the positional Skiff tuple to table columns. rows is checked against that schema before the request is made.

Errors

Returns ClientError if the format is not a direct-table format, the stream is incomplete, or the request fails.

pub fn read_table(&self, path: impl Into<TablePath>) -> Result<Vec<u8>>

Reads a whole table as a binary YSON list fragment.

Reads it into memory: this is for results a launcher inspects, not for bulk export.

The path can select which part of the table to read — TablePath::columns and TablePath::range travel as attributes on it, so three columns of a hundred rows cost three columns of a hundred rows, not the whole table:

let head = client.read_table(TablePath::new("//tmp/log").columns(["host"]).range(0..100))?;

The result is checked to be a complete list fragment. That is not pedantry — the proxy reports a mid-stream failure in a trailer this client cannot see (see the http module), so a truncated body is the symptom that is detectable, and returning it as success would hand the caller a silently short table.

Errors

Returns ClientError if the request fails or the stream is truncated.

pub fn read_table_with_format( &self, path: impl Into<TablePath>, format: &DataFormat, ) -> Result<Vec<u8>>

Reads a whole table using a shared DataFormat.

The returned bytes are a YSON list fragment or a complete Skiff stream, according to format. The response is checked for truncated records before it is returned.

Errors

Returns ClientError if the format is unsupported, the response is incomplete, or the request fails.

pub fn read_skiff_table( &self, path: impl Into<TablePath>, format: &SkiffFormat, ) -> Result<Vec<u8>>

Reads one table as a complete Skiff stream.

format must have exactly one table schema. Its named fields select the table columns and determine the bytes returned — which is why a path that also names columns is refused. That covers both spellings, TablePath::columns and {…} in the path string, because the format’s fields become a columns attribute here whether the caller named one or not.

What that costs is a silently ignored filter, not a corrupt decode. Measured, the synthesised attribute wins: <columns=[n]>"//tmp/t{k}" answered with column n. A Skiff read therefore still receives exactly the columns its format names, and the tuple stays aligned — but the {…} the caller wrote is discarded without a word, at 200. Refusing is how they get to hear about it. A path string opening with <…> is refused one step removed: this client cannot parse the block to see whether it names columns as well.

Row selections are not column selections and are not refused. A TablePath::range combines, and so does a range spelled into the string — measured, <columns=[n]>"//tmp/t[#0:#2]" answered 200 with rows 0-1 carrying only n. Ranges pick rows, the schema picks columns.

The response is decoded to its end before being returned so a truncated Skiff stream is never reported as a successful table read.

Errors

Returns ClientError if the format is not a direct-table format, the path also selects columns — through TablePath::columns or as {…} in its string — the path string opens with an attribute block, the response is incomplete, or the request fails.

pub fn write_table_rows<T, I>( &self, path: impl Into<TablePath>, rows: I, ) -> Result<()>

where T: Serialize, I: IntoIterator<Item = T>,

Writes rows to a table from anything that yields them.

The rows are Rust values; the encoding is this crate’s problem, which is the difference between this and Client::write_table:

#[derive(serde::Serialize)]
struct Contact<'a> {
    name: &'a str,
    email: &'a str,
    age: i64,
}

client.write_table_rows("//tmp/contacts", (0..100).map(|n| Contact {
    name: "Gordon Freeman",
    email: "gordon@black-mesa.example",
    age: 27 + n,
}))?;

It takes an iterator rather than a slice because the encoder sits inside the request body: rows are serialised a bufferful at a time as the connection asks for bytes, so a million rows cost one buffer rather than a million rows’ worth of memory, and the caller never has to materialise them either.

Replaces the table’s contents, as Client::write_table does — and refuses a path carrying a read selection before anything is sent, for the reason given there.

Errors

Returns ClientError::Config if the path carries a read selection, ClientError::Decode naming the row if one cannot be serialised — the write fails rather than sending the rows before it — or ClientError if the request fails.

pub fn read_table_rows<T: DeserializeOwned>( &self, path: impl Into<TablePath>, ) -> Result<Vec<T>>

Reads a whole table as typed rows.

#[derive(serde::Deserialize)]
struct Contact {
    name: String,
    age: i64,
}

for contact in client.read_table_rows::<Contact>("//tmp/contacts")? {
    println!("{} is {}", contact.name, contact.age);
}

Rows are owned, and the whole table is read before any of it is returned — this is Client::read_table with the decoding done, and it inherits the same purpose: results a launcher inspects. For a table that does not fit, or for rows borrowed from the buffer they arrived in, Client::read_table_streaming feeds ytsaurus_job::JobReader.

Columns the type does not mention are ignored, so a struct naming two columns of a twenty-column table is a projection rather than an error — but the whole row still crosses the wire and is decoded before the projection happens. TablePath::columns moves the projection to the cluster, and TablePath::range does the same for rows:

let some: Vec<Contact> = client.read_table_rows(
    TablePath::new("//tmp/contacts").columns(["name", "age"]).range(0..100),
)?;

Errors

Returns ClientError if the request fails, the stream is truncated, or a row does not match T.

pub fn get_as<T: DeserializeOwned>(&self, path: &str) -> Result<T>

Reads a node, or an attribute, into a Rust type.

Client::get hands back a YsonValue to walk; this hands back the shape you were going to walk it into:

#[derive(serde::Deserialize)]
struct Cluster {
    #[serde(rename = "type")]
    node_type: String,
    creation_time: String,
    account: String,
}

let root: Cluster = client.get_as("//@")?;
println!("the cluster was created at {}", root.creation_time);

Attributes the type does not mention are ignored, which is what makes //@ — a node with dozens of them — worth asking about at all.

Errors

Returns ClientError if the request fails or the answer does not fit T.

pub fn read_table_streaming( &self, path: impl Into<TablePath>, ) -> Result<TableReader>

Reads a table as a stream, without holding it.

The same bytes Client::read_table returns — a binary YSON list fragment — arriving as they come off the connection, so the table’s size stops being the program’s memory ceiling.

What comes out is what a job reads on fd 0, so the same decoder handles both:

let mut reader = ytsaurus_job::JobReader::binary(client.read_table_streaming("//tmp/big")?);

let mut rows = 0_u64;
while let Some(event) = reader.next_event()? {
    if matches!(event, ytsaurus_job::Event::Row(_)) {
        rows += 1;
    }
}

Client::read_table checks that what came back is a complete fragment; this cannot, because it never has the whole thing. A fragment cut short instead leaves a record that does not parse, and the decoder fails on it — see TableReader for why that is the same protection rather than none.

The path can carry a read selection — TablePath::columns and TablePath::range — which is worth the most here of anywhere: a streaming read exists because the table is too big to hold, and a selection is how most of it never arrives at all.

Errors

Returns ClientError if the request fails. Failures during the read arrive from the reader, not from here.

pub fn write_table_streaming( &self, path: impl Into<TablePath>, rows: impl Read, ) -> Result<()>

Writes a table from a stream, without holding it.

rows is read to its end and sent as it is read, so the rows can come from a file, a pipe, or something that generates them — anything that is a Read. The bytes are a binary YSON list fragment, exactly as Client::write_table expects them.

client.create("table", "//tmp/big")?;
client.write_table_streaming("//tmp/big", std::fs::File::open("rows.yson")?)?;

This is one attempt and can never be more: a reader that has been consumed cannot be sent again. That agrees with the retry rules — heavy commands are not repeated — and a transaction is what makes such a write safe to fail.

Errors

Returns ClientError::Config if the path carries a read selection — see Client::write_table — or ClientError if the request fails, including when rows itself fails to read.

pub fn start_map(&self, spec: &MapSpec) -> Result<String>

Starts a map operation, returning its ID.

Errors

Returns ClientError if the request fails.

pub fn start_map_reduce(&self, spec: &MapReduceSpec) -> Result<String>

Starts a map-reduce operation, returning its ID.

Errors

Returns ClientError if the request fails.

pub fn start_reduce(&self, spec: &ReduceSpec) -> Result<String>

Starts a reduce operation over sorted input, returning its ID.

The input tables must already be sorted by a column set beginning with the spec’s reduce_by; the cluster refuses the operation otherwise. Client::start_sort is how they get that way.

Errors

Returns ClientError if the request fails.

pub fn start_sort(&self, spec: &SortSpec) -> Result<String>

Starts a sort operation, returning its ID.

Errors

Returns ClientError if the request fails.

pub fn start_vanilla(&self, spec: &VanillaSpec) -> Result<String>

Starts a vanilla operation, returning its ID.

Jobs with no input tables: a distributed process, a side-car computation, anything that is not a transformation of a table.

Errors

Returns ClientError::Config if two tasks share a name, and ClientError if the request fails.

pub fn start_merge(&self, spec: &MergeSpec) -> Result<String>

Starts a merge operation, returning its ID.

A MergeMode::Sorted merge does not need MergeSpec::with_merge_by: measured against a cluster, one sent without it is accepted and the key is taken from the sort columns the inputs already carry, with the output coming back sorted by them. Naming the columns is how to merge by fewer of them than the inputs are sorted by, or to state the assumption where a reader can see it.

Errors

Returns ClientError if the request fails — including when a sorted merge’s inputs are not sorted, which only the cluster can tell.

pub fn start_erase(&self, spec: &EraseSpec) -> Result<String>

Starts an erase operation, returning its ID.

Errors

Returns ClientError if the request fails.

pub fn start_remote_copy(&self, spec: &RemoteCopySpec) -> Result<String>

Starts a remote-copy operation, returning its ID.

Errors

Returns ClientError if the request fails.

pub fn start_operation( &self, kind: OperationType, spec: &YsonValue, ) -> Result<String>

Starts an operation from a spec built by hand.

The escape hatch for anything MapSpec and MapReduceSpec do not model; build the spec with yson_build.

Errors

Returns ClientError if the request fails.

pub fn start_operation_with( &self, kind: OperationType, spec: &YsonValue, mutation_id: &MutationId, ) -> Result<String>

Starts an operation under a mutation ID you control.

start_operation already tags its own retries with a fresh MutationId, so a retried start never leaves two operations running. This is for the guarantee a single process cannot give itself: persist the ID, and after a crash the same call returns the operation that was already started instead of starting a second one.

The cluster remembers a mutation ID for five to ten minutes, so this is a guard against a crash-and-restart, not a permanent key.

Errors

Returns ClientError if the request fails.

pub fn abort_operation(&self, id: &str, reason: Option<&str>) -> Result<()>

Stops an operation that is still running.

The counterpart to starting one, and the reason it is worth having: a launcher that gives up — an interrupted wait_for_operation, a failed step further down the script — otherwise leaves the operation running on the cluster, spending quota on a result nobody will read.

reason is put in the operation’s error document, under the cluster’s own Operation aborted by user request, so whoever finds the aborted operation later is told who stopped it and why. Pass None to say nothing.

By the time this returns the operation is already aborted: the call takes a few hundred milliseconds, and the state has changed within it. The aborting state exists but no caller of this can observe it.

This is not idempotent, unlike Transaction::abort. Once the scheduler has let go of an operation it answers No such operation, and it lets go as soon as the first abort is accepted — so a second abort is an error rather than a shrug, even for an operation that was still running a moment ago. An operation that finished by itself can still be aborted for the short while the scheduler keeps it, so this is not a reliable way to ask whether one has finished either.

Sent once, and never retried, which is the other side of the same coin. abort_operation is a scheduler command and the master’s mutation cache does not cover it: a retry after a lost answer would be told No such operation and would report a successful abort as a failed one. A transport error here means the request may or may not have arrived, and the honest thing is to say so rather than to guess.

Errors

Returns ClientError if the request fails, including when the scheduler no longer has the operation.

pub fn suspend_operation( &self, id: &str, abort_running_jobs: bool, ) -> Result<()>

Pauses a running operation.

Its jobs stop being scheduled; what is already running keeps running unless abort_running_jobs says otherwise, in which case the work those jobs had done is lost and will be done again after Client::resume_operation.

Suspension is not a state. A suspended operation still answers running to Client::operation_state — the cluster reports it in a separate suspended attribute, which is what Client::operation_suspended reads. Verified on a local cluster, and it is the sort of thing a poll loop gets wrong forever.

Unlike its counterpart, this one is idempotent: suspending a suspended operation answers {}, so it is retried like a read. That holds only while the scheduler still has the operation — once it has let go, this answers No such operation like every other command here.

Errors

Returns ClientError if the request fails, including when the scheduler no longer has the operation.

pub fn resume_operation(&self, id: &str) -> Result<()>

Lets a suspended operation run again.

Sent once, and never retried. Where Client::suspend_operation is idempotent, this is not: an operation that is not suspended answers code 201, Operation is in "running" state. A retry after a lost answer would therefore report a resume that worked as a failure — the same trap Client::abort_operation describes.

Errors

Returns ClientError if the request fails, including when the operation was not suspended.

pub fn complete_operation(&self, id: &str) -> Result<()>

Finishes an operation early, keeping what it has produced.

The difference from Client::abort_operation: an aborted operation’s output tables are discarded, a completed one’s are published. This is how a long-running vanilla operation is stopped successfully — it ends as completed, and Client::wait_for_operation returns Ok.

Sent once, and never retried, for the reason Client::abort_operation gives: the second one is answered No such operation, so a retry turns a completion that worked into an error.

Errors

Returns ClientError if the request fails, including when the scheduler no longer has the operation.

pub fn update_operation_parameters( &self, id: &str, parameters: &OperationParameters, ) -> Result<()>

Changes a running operation’s scheduling parameters.

The pool it competes in and the share it gets, while it runs — the one thing about a started operation that is not fixed. See OperationParameters.

client.update_operation_parameters(
    &id,
    &OperationParameters::new().with_pool("interactive").with_weight(2.0),
)?;

The parameters go in the request’s parameters, not its body: the cluster’s registry declares this command’s input as null, whatever the command reference says. It answers with an empty body rather than the {} its neighbours send.

Repeated freely, because it assigns rather than increments: sending the same update twice leaves the operation where the first one put it. As with Client::suspend_operation, that holds only while the scheduler still has the operation — if the answer to the first send is lost and the operation ends during the backoff, the retry is answered No such operation and this returns an error for an update that was applied.

Errors

Returns ClientError::Config if parameters would change nothing — the cluster accepts an empty update and does nothing, which hides the mistake where it was made — and ClientError if the request fails.

pub fn list_operations(&self, filter: &OperationFilter) -> Result<OperationList>

Lists operations the cluster knows about.

let mine = client.list_operations(
    &OperationFilter::new().with_user("robot-loader").with_state("running"),
)?;

for operation in &mine.operations {
    println!("{} {} {}", operation.id, operation.kind, operation.state);
}

The scheduler only holds operations it has not let go of. Anything older lives in the operations archive, which OperationFilter::with_archive asks for — and which a local cluster does not have.

Errors

Returns ClientError if the request fails or the response cannot be decoded.

pub fn list_operation_events(&self, id: &str) -> Result<Vec<OperationEvent>>

An operation’s event log.

Empty on a cluster with no operations archive. The command is registered everywhere and answers with an empty list there, rather than with an error — verified on a local cluster, where it is always empty.

Errors

Returns ClientError if the request fails or the response cannot be decoded.

pub fn attach_operation(&self, id: impl Into<String>) -> Operation

A handle on an operation that is already running.

The reattach door — C++’s AttachOperation, Go’s Track(id). Nothing is sent: an id and a client is all an Operation is, so this cannot fail and does not check that the operation exists. The first command through the handle finds that out.

// A supervisor restarts and picks up where it left off.
let op = client.attach_operation(std::fs::read_to_string("run.id")?);
op.wait()?;

The id is trimmed, for the reason the token file is: the documented way to get one here is out of a file, echo $ID > run.id writes a newline, and an id carrying one is answered No such operation by an error that never mentions whitespace.

pub fn get_operation(&self, id: &str, attributes: &[&str]) -> Result<YsonValue>

The whole document the cluster keeps about an operation.

attributes names what to fetch — state, progress, result, runtime_parameters, spec. An empty slice asks for everything, which is rarely what anyone wants: the full document for a trivial vanilla operation measured 119 KB on a local cluster, most of it the resolved spec and the progress tree. Naming attributes is the normal case, and the narrow readers — Client::operation_state, Client::job_statistics, Client::operation_result_error — are each one attribute of this.

let doc = client.get_operation(&id, &["state", "start_time", "suspended"])?;

Errors

Returns ClientError if the request fails or the answer cannot be decoded.

pub fn get_operation_by_alias( &self, alias: &str, attributes: &[&str], ) -> Result<YsonValue>

The same, for an operation found by the alias its spec gave it.

An alias is a name a launcher chooses — *nightly-load — set in the spec’s alias field, and the leading * is the cluster’s requirement, not this crate’s. Without it, an alias set at launch could never be looked up again.

The request carries include_runtime, because the cluster refuses the lookup without it: “Operation alias cannot be resolved without using runtime information”. That also bounds what this can find — an alias is resolved from what the scheduler still holds, falling back to the operations archive, so an alias whose operation finished long ago is found only on an installation that has an archive.

Errors

Returns ClientError if the request fails — including when no operation has that alias — or if the answer cannot be decoded.

pub fn operation_state(&self, id: &str) -> Result<String>

Fetches an operation’s current state, e.g. running or completed.

A suspended operation still reports running. See Client::operation_suspended, or Client::operation_status for both in one request.

Errors

Returns ClientError if the request fails.

pub fn operation_suspended(&self, id: &str) -> Result<bool>

Whether an operation is paused.

The question Client::operation_state does not answer: the cluster keeps suspension in its own attribute and leaves the state at running, so a loop that watches the state alone will wait out a paused operation without ever saying why.

An operation whose document does not carry the attribute is not suspended, rather than an error: the scheduler reports it for what it still holds, and one resolved out of the operations archive may not carry it at all.

Errors

Returns ClientError if the request fails, or if the attribute is there and is not a boolean.

pub fn operation_status(&self, id: &str) -> Result<OperationStatus>

An operation’s state and whether it is paused, in one request.

The pair a poll loop actually needs. Asking them separately is two round trips for two attributes of one document, and a loop that asks only for the state cannot tell a running operation from a paused one — they both say running.

let status = client.operation_status(&id)?;
if status.suspended {
    println!("paused — it will sit at {} until it is resumed", status.state);
}

Errors

Returns ClientError if the request fails or the answer cannot be decoded.

pub fn custom_statistics(&self, operation_id: &str) -> Result<YsonValue>

The custom statistics an operation’s jobs reported.

Returns the custom subtree of the operation’s job statistics, keyed by the names the jobs used. Each leaf is an aggregate — sum, count, min, max — over the jobs that reported it, so a per-row counter comes back as one number for the whole operation. Client::statistic_sum pulls a single total out of it.

Empty if no job reported anything.

Errors

Returns ClientError if the request fails.

pub fn job_statistics(&self, operation_id: &str) -> Result<YsonValue>

Everything the scheduler recorded about an operation’s jobs.

The whole job_statistics tree, custom and built-in alike. Client::job_statistic_sum is the way to read one number out of it; this is for looking around, which is how anyone finds out what a cluster actually reports.

Errors

Returns ClientError if the request fails.

pub fn job_statistic_sum( &self, operation_id: &str, path: &str, ) -> Result<Option<i64>>

The total of one built-in job statistic, e.g. time/exec.

The cluster’s own statistics nest by path component, where a custom name keeps its slash as one key — the two are stored differently, which is why they are read differently:

custom:    {"rows/rejected" = {"$"  = {completed = {map = {sum=3}}}}}
built-in:  {time = {exec    = {"$$" = {completed = {map = {sum=744}}}}}}

Note the separator differs too — $$ rather than $. Both are accepted here, because that difference is not something a caller should have to know.

Totalled over completed jobs across job types, as Client::statistic_sum does, and None when the cluster reports nothing under that path — which is not the same as zero. A local cluster reports nothing under user_job/cpu, for instance.

Errors

Returns ClientError if the request fails.

pub fn statistic_sum( &self, operation_id: &str, name: &str, ) -> Result<Option<i64>>

The total of one custom statistic over an operation’s completed jobs.

name is exactly what the job called it, slashes included: the cluster keeps rows/rejected as one key rather than nesting it.

Only completed jobs are counted. An aborted job’s work is done again by its replacement, so including it would count the same rows twice. Job types are summed together, so a map-reduce reporting one name from both phases gives the operation’s total.

None means no job reported that name — which is not the same as zero.

Errors

Returns ClientError if the request fails.

pub fn wait_for_operation(&self, id: &str) -> Result<()>

Polls until the operation reaches a terminal state.

A suspended operation never reaches one, and this says so rather than sitting there: suspension is not a state, so a paused operation goes on answering running for as long as it is paused. The progress line reports it, which is the difference between a wait that looks hung and one that names what it is waiting for. Resuming it — from another process, or from the one that paused it — is what ends the wait.

Errors

Returns ClientError::OperationFailed if it ends as anything other than completed, or ClientError if polling itself fails.

pub fn operation_result_error(&self, id: &str) -> Result<Option<String>>

Why an operation ended as it did, in the cluster’s words.

None for one that succeeded, and for one that has not finished. This is what ClientError::OperationFailed carries, and what reads back the reason given to Client::abort_operation: the reason is folded into the operation’s error document rather than kept beside it, so this is how to find out who stopped an operation and why.

Flattened to the outer message plus the innermost one, because the outer message of a YTsaurus error is a category and the cause is at the bottom.

Errors

Returns ClientError if the operation cannot be looked up, or if its answer cannot be decoded.

pub fn list_jobs( &self, operation_id: &str, state: Option<&str>, limit: u32, ) -> Result<Vec<JobInfo>>

Lists an operation’s jobs.

state filters by job state — failed, completed, running, … — and limit caps how many come back.

The YTsaurus documentation warns that list_jobs can put significant load on a cluster and asks that it not be part of a workflow without an administrator’s approval. This client calls it once per failed operation, with a small limit; keep to that shape.

Errors

Returns ClientError if the request fails or the response is not the documented {jobs=[…]}.

pub fn get_job(&self, operation_id: &str, job_id: &str) -> Result<JobInfo>

Fetches one job of an operation.

What Client::list_jobs reports for a job it lists, asked for by id — and the way to look at a job whose id came from somewhere else, a log line or the web interface, without listing every job of the operation.

The cluster answers with the job document unwrapped, and calls the id job_id where list_jobs calls it id; both are read here, so the JobInfo that comes back is the same shape either way.

Errors

Returns ClientError if the request fails, or if the answer names no job — which is what an unknown job id looks like.

pub fn get_job_input( &self, operation_id: &str, job_id: &str, ) -> Result<ResponseReader>

Streams the input a job was given.

The rows the cluster fed to that one job, in the format its spec asked for — which is how a job that failed on one row is reproduced on a desk rather than on the cluster.

This is a heavy command whose answer is the data, so it streams: nothing here holds the job’s input, and on an installation that separates light and heavy proxies it is sent to the heavy one.

A job with no input never answers. Measured against a local cluster: the request for a vanilla job’s input sat for 30 seconds without a byte. A vanilla operation has no input tables, so there is nothing for the cluster to send and it does not say so; ask this only of a job that reads something.

Errors

Returns ClientError if the request fails. Failures during the read arrive from the reader, for the reason ResponseReader describes.

pub fn get_job_stderr( &self, operation_id: &str, job_id: &str, ) -> Result<Vec<u8>>

Fetches what a job wrote to stderr.

Returns raw bytes: stderr is whatever the process wrote, not necessarily UTF-8. Empty if the cluster saved nothing — stderr is kept for failed jobs and, when the spec asks for it, for successful ones.

This is a heavy command, so on an installation that separates light and heavy proxies it goes to the heavy one, like a table read.

Errors

Returns ClientError if the request fails.

pub fn raw_command( &self, method: Method, command: &str, params: &YsonValue, payload: Option<&[u8]>, ) -> Result<Vec<u8>>

Sends a command this crate does not model, and hands back the answer.

Every other method here is a command the crate has an opinion about: parameters built for you, the response decoded into a type. This is the door to the rest of API v4 — the commands this crate has not grown yet, and the ones it never will. It is the same door Client::start_operation opens for a hand-built spec, widened from one command to all of them, and it means the answer to “can I do X against my cluster?” stops being “fork the crate”.

params is the X-YT-Parameters dict — build it with yson_build. payload is the request body, for a command that takes one. What comes back is the response body, exactly as the proxy sent it; API v4 wraps a structured answer in a one-key dict, so most commands answer {key=…} in text YSON.

let client = Client::from_env()?;

// `get_supported_features` is not modelled here and takes no
// parameters. It answers with what this cluster's build can do —
// codecs, compression, primitive types — which is exactly the question
// a crate that models a quarter of the API cannot answer for you.
let body = client.raw_command(
    Method::Get,
    "get_supported_features",
    &yson_build::empty_map(),
    None,
)?;

println!("{}", String::from_utf8_lossy(&body));

What this still does for you

Everything that is not about the command’s meaning: the token, the timeout, TLS, the header encoding, the X-YT-Error check that turns a cluster failure into a ClientError::Cluster with the innermost message — and the client’s transaction. A raw command is stamped with transaction_id like every other, so a command sent through Transaction is in that transaction rather than quietly outside it. The exceptions are the same: a command that names its own transaction keeps it, and the scheduler commands are not stamped at all.

What it does not

It is sent once, and to the configured address. A command this crate does not model cannot be assumed non-mutating, and a retry that applied an unknown mutation twice would be a far worse failure than one lost to a flaky proxy — so the default is Repeatable::Never and the retry policy is ignored here, whatever it says.

Never is the safe answer for repeating, and it is the wrong answer for routing: it sends the command to the address the client was configured with, which on an installation that separates proxy roles is a control proxy that will not serve a heavy one. A raw write_file sent this way is refused with Control proxy may not serve heavy requests with input data, and a raw read_file is answered with a 307 to a data proxy. Client::raw_command_with is where a caller who knows the command is heavy says Repeatable::Heavy and gets both halves of that answer at once.

The streaming doors need no such care: Client::raw_command_streaming and Client::raw_command_upload are heavy by construction, because streaming is the heavy shape.

Nor does it know the verb: see Method for the cluster’s own rule for picking one.

Errors

Returns ClientError::Config if command is not a bare command name, if params is not a YSON dict — every command’s parameters are one, and the client adds to them — or if a body is passed with Method::Get, which carries none, so it would be dropped in silence. Otherwise ClientError as any command fails.

pub fn raw_command_with( &self, method: Method, command: &str, params: &YsonValue, payload: Option<&[u8]>, repeatable: Repeatable, mutation_id: Option<&MutationId>, ) -> Result<Vec<u8>>

As Client::raw_command, saying how the command may be repeated.

The judgement this needs is the cluster’s, not a guess: a command declares whether it mutates and whether it is heavy, and Repeatable is how that reaches the retry policy. Repeatable::Freely for a read, Repeatable::WithMutationId for a light mutation the master’s mutation cache covers, Repeatable::Heavy for one that moves table or file data — which also sends it to a proxy that will accept one — Repeatable::Never otherwise.

“Light and mutating” is not by itself enough for a mutation ID: the cache lives in the master, and a command that goes to the scheduler is not covered by it. Verified for abort_operation — a second send of the same ID, flagged as a retry, is answered No such operation rather than with the first response, so the retry turns an abort that worked into an error the caller believes. Whether every scheduler command behaves that way was not checked; treat it as the working assumption and prefer Never when in doubt.

mutation_id is for the guarantee a single process cannot give itself: persist it, and after a crash the same call is deduplicated against the one that already ran instead of applying twice. See MutationId.

An ID given here is stamped on the request whatever repeatable says, including under Repeatable::Never — the two answer different questions. repeatable decides whether this call may be sent twice; a mutation ID decides whether a later call, from a process that has since restarted, is recognised as the same mutation. A command that must not be retried in-process can still be worth making replayable across one, and this is how.

Errors

As Client::raw_command.

pub fn raw_command_streaming( &self, method: Method, command: &str, params: &YsonValue, ) -> Result<ResponseReader>

Sends a command this crate does not model and hands back its response unread.

For a command whose answer is the data — read_blob_table, anything the cluster declares heavy on the way out. Client::raw_command would put all of it in memory first, which for those is the thing worth avoiding.

// `read_file` has a method now — `Client::read_file_streaming` is
// this call with the parameters written down — and it stays as the
// example because its wire shape is verified against a cluster, where
// an unmodelled command's here would be a guess. The door sends any
// command the same way.
let mut file = client.raw_command_streaming(
    Method::Get,
    "read_file",
    &yson_build::map([("path", yson_build::string("//tmp/worker"))]),
)?;

std::io::copy(&mut file, &mut std::fs::File::create("worker")?)?;

Sent once, and never retried: this is the shape a heavy command takes, and the documentation is explicit that heavy commands are not repeated. It is also sent to a heavy proxy, for the same reason and without asking — a response that is the data is Repeatable::Heavy whatever the command turns out to be called. The request carries no body — Client::raw_command_upload is the other direction.

The streaming timeout applies, so the transfer itself is not on the request clock; see Client::with_timeout.

Errors

Returns ClientError::Config if command is not a bare command name, and ClientError if the request fails. Failures during the read arrive from the reader, not from here — and a body cut short by a mid-stream failure ends quietly, for the reason ResponseReader describes.

pub fn raw_command_upload( &self, method: Method, command: &str, params: &YsonValue, body: impl Read, ) -> Result<Vec<u8>>

Sends a command this crate does not model, streaming its request body.

The counterpart of Client::raw_command_streaming, for a command that takes an input data stream — the PUT commands, in the cluster’s own rule. body is read to its end and sent as it is read, so what is uploaded never has to fit in memory.

This is one attempt and can never be more: a reader that has been consumed cannot be sent again. A transaction is what makes such a write safe to fail. And it goes to a heavy proxy, as Client::raw_command_streaming does and for the same reason.

Errors

Returns ClientError::Config if command is not a bare command name, or if the verb is Method::Get, which carries no body. Otherwise ClientError if the request fails, including when body itself fails to read.

Trait Implementations

impl Debug for Transaction

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Deref for Transaction

type Target = Client

The resulting type after dereferencing.

fn deref(&self) -> &Client

Dereferences the value.

impl Drop for Transaction

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.