Module chunked 

Op-layer for the chunkedTrustTask backup transfer algorithm.

The stream algorithm moves a bundle over the VTA’s HTTPS blob endpoint, which a VTA reachable only over DIDComm or TSP does not have. This one moves it as a sequence of Trust Tasks over whatever transport carries the control plane: vta/backup/get-chunk/1.0 pulls an export chunk by index, and vta/backup/put-chunk/1.0 writes an import chunk checked against a manifest the operator committed before any chunk moved. The normative account is vta/backup/initiate-export/1.1 § Chunked transfer (trustoverip/dtgwg-trust-tasks-tf#474).

It reuses the descriptor pattern’s BundleRecord state machine and staging directory unchanged, and keeps what is new — the manifest and which indices have moved — in a ChunkPlan beside the record, under its own key prefix, so the record’s shape (and every reader of it) is untouched.

Four properties the specification makes normative, and where each is held:

• Pulled, not pushed. Nothing here sends; every chunk is an answer to a request, so a chunk lost in transit is one the client knows it lacks.
• Non-consuming reads. get_chunk serves by offset from the staged file and never deletes it; the bundle goes on complete-export, abort or expiry.
• Idempotent writes. put_chunk checks every chunk against the pre-committed digest, so the only bytes an index can ever hold are the committed ones; a repeat is stored: false.
• Bounded expiry. Activity slides expires_at forward, never past a ceiling fixed when the bundle was minted.

Structs

ChunkPlan

The manifest and progress of one chunked bundle, stored at chunks:{bundle_id} in the bundles keyspace beside its BundleRecord.

ChunkRateLimiter

Token bucket per authenticated DID for chunk requests.

ChunkWrite

One put-chunk request, as the op reads it.

ChunkedBundle

What a chunked initiate-export hands back: the manifest and the terms.

PutOutcome

What one accepted write reports back.

ServedChunk

One served chunk.

Enums

ChunkedError

Why a chunked operation refused. Kept apart from AppError because most arms are a specification error code the handler must put on the wire as <task-slug>:<code>, and several carry the details that code declares.

Constants

CHUNK_REQUESTS_PER_SECOND

Chunk requests one DID may make per second, sustained.

CHUNK_REQUEST_BURST

Burst above CHUNK_REQUESTS_PER_SECOND a DID may spend at once.

MAX_REPORTED_MISSING

Largest missingIndices list an incompleteUpload refusal carries (vta/backup/finalize-import/1.1 bounds it).

Functions

all_served

Whether every chunk of a chunked export has been served at least once — the meaning complete-export’s downloaded takes for such a bundle. None for a bundle with no chunk plan (a stream bundle).

delete_plan

Remove a bundle’s chunk plan. Called when the bundle ends and by the sweeper’s retention pass; absent is not an error.

finalize_precheck

The checks finalize-import must make of a chunked bundle before the password is used: every chunk present, and the assembled bytes equal to the committed whole-bundle digest and size. On success the bundle moves to ImportReceived, which is the state the finalize op accepts.

get_chunk

Serve chunk index of a chunked export bundle.

get_plan

Fetch the chunk plan for a bundle, if it has one.

initiate_export

Mint a chunkedTrustTask export bundle.

initiate_import

Open a chunked import slot for a manifest the producer has pre-committed.

put_chunk

Write chunk index of a chunked import.

stage_export

Stage already-encrypted export bytes as a chunked bundle. Split from initiate_export so the staging and serving logic can be exercised without a full agent to serialize.