Module project_bundle 

The BRPROJ project export/import bundle format (tar + JSON MANIFEST + per-member SHA-256). Dependency-light + testable in every build; the driver that fills it is feature-gated. The BRPROJ project export/import bundle — the on-disk format a project export writes and a project import reads.

A bundle is a plain tar whose FIRST entry is a JSON MANIFEST describing every other member, followed by the per-engine payloads:

MANIFEST                               magic "BRPROJ" | format_version | created_at | data_only | members[]
kv.brkvdump                            the control-plane KV logical slice (kv_dump::export_to_bytes bytes)
sql/<db>.pgc                           a streamed Postgres dump (pg_dump custom/plain)
libsql/<project>/<site>/<db>.sqlite    a VACUUM INTO snapshot of a libsql database
blobs/<...>                            blob-store objects

Discipline borrowed from boatramp_core::kv_dump: a loud magic (BRPROJ) and an explicit format version the reader refuses if unknown (never misreads a foreign/newer file), plus a per-member SHA-256 checksum the reader VERIFIES on extraction (byte-integrity, not a size check) — so a truncated/corrupt bundle is caught before a single row is restored.

The writer takes members already staged on disk (the multi-GB pg_dump stream lands in a staging file via run_ephemeral’s streaming sink, the libsql VACUUM INTO writes a file, the KV slice + blobs are staged too), so each member is appended with a known size — correctness over the PAX unknown-length streaming form, and the staging keeps memory bounded regardless of dump size. All IO is synchronous (the tar crate is sync); the async driver calls it from a blocking task.

Structs

BundleStreamWriter

A streaming v2 bundle writer: writes the MANIFEST up front, then appends members ONE AT A TIME (append_member), each from a bounded staging temp the caller may delete the instant the call returns — so the node never holds more than the largest single member. The export driver drives it from a blocking task, staging each member just before it appends and deleting the temp just after (see the convenience wrapper write_bundle_streaming, which encapsulates exactly that produce→append→delete loop).

Manifest

The bundle MANIFEST — the first tar entry, describing the whole bundle.

ManifestMember

One member described in the MANIFEST.

MemberMeta

A member’s .brmeta sidecar: the size + SHA-256 the v1 MANIFEST used to carry inline, now written immediately before the member so the reader knows them before the body.

MemberSpec

The upfront-knowable identity of a bundle member — what the v2/v3 MANIFEST lists, in restore order. Carries no size/SHA (those travel in the member’s MemberMeta sidecar or MemberTrailer), so the MANIFEST can be written before any member is produced.

MemberTrailer

A chunked member’s .brtrailer: the size + SHA-256 of the whole member, written AFTER its .part{N} chunks so a streaming producer need never stat or buffer the member (the chunked analogue of the v2 leading MemberMeta).

PgMemberInfo

SQL-dump provenance for a PgDump/MysqlDump member: the detected server version, the client image used to produce (and required to restore) it, and the schema-ledger fingerprint for the import schema-skew check. (Field named pg historically; it carries MySQL provenance too — the engine is the member’s MemberKind.)

SourceInfo

What the bundle was exported from.

StagedMember

A member staged on disk, ready to be appended to a bundle.

StreamManifest

The v2 MANIFEST — the first tar entry: the ordered member LIST plus bundle metadata. Unlike the v1 Manifest it carries NO per-member size/SHA (each member’s live in its MemberMeta sidecar), which is what makes a streaming export possible.

Enums

BundleError

A project-bundle error.

MemberFraming

How a bundle member’s bytes are framed in the tar. Sidecar (the default, and the only v2 framing) is a leading MemberMeta .brmeta then a single body entry; ChunkedTrailer is a run of .part{N} entries then a trailing MemberTrailer .brtrailer (v3).

MemberKind

Which engine a bundle member came from.

Constants

BUNDLE_FORMAT_VERSION

The bundle FORMAT version. Bumped only on an incompatible change; a reader refuses an unknown version rather than misreading it.

BUNDLE_FORMAT_VERSION_V2

The streamable bundle format version (see the section comment above).

BUNDLE_FORMAT_VERSION_V3

The chunked bundle format version (see the section comment above). Emitted when at least one member uses MemberFraming::ChunkedTrailer; a v3 reader reads v1 + v2 + v3.

BUNDLE_MAGIC

The bundle magic — carried in the MANIFEST so a stray/foreign tar is rejected loud.

IDENTITY_MEMBER_PATH

The identity transport-container’s tar-entry path (--include-identity only): the passphrase-wrapped KEK, produced by boatramp_server::envelope::transport_wrap.

KV_MEMBER_PATH

The control-plane KV slice’s tar-entry path (the kv_dump::export_to_bytes bytes).

MANIFEST_PATH

The manifest’s tar-entry path — MUST be the first entry in the bundle.

META_SUFFIX

The suffix of a member’s sidecar tar entry. For a data member at <path>, its sidecar is <path>.brmeta — a tiny JSON MemberMeta (size + SHA-256) written immediately before <path>, so a single-pass reader learns them before the body arrives.

PART_INFIX

The infix of a chunked member’s data-part tar entries. Chunk N of a member at <path> is the entry <path>.part{N:05} (zero-padded, monotonic from 00000).

TRAILER_SUFFIX

The suffix of a chunked member’s TRAILING trailer entry. For a chunked member at <path>, its trailer is <path>.brtrailer — a tiny JSON MemberTrailer (size + SHA-256) written AFTER the member’s .part{N} chunks, so the reader verifies integrity once the whole member has streamed (the chunked analogue of the v2 leading MemberMeta sidecar).

Traits

BundleRestorer

The restore policy the v3-capable single-pass reader read_bundle_streaming_atomic drives, one method per member shape. The format layer owns the PARSE + the per-member integrity VERIFY; the implementor (the import driver) owns the APPLY (where a member is re-keyed to the target, scope-enforced, and committed) — so a chunked member and a sidecar member of the same kind route through the SAME driver re-key/enforce, framing being a transport detail, never a tenancy bypass.

Functions

extract_and_verify

Extract every member of bundle into out_dir, VERIFYING each member’s SHA-256 against the MANIFEST as it streams (byte-integrity). Refuses a member the manifest does not describe, and a manifest member the bundle does not contain. Returns the validated MANIFEST. Member files land at out_dir/<member.path>.

read_bundle_streaming

Read a bundle from src in a SINGLE FORWARD PASS, handing each member to restore as it streams — never staging the whole bundle. Reads the MANIFEST first, then for each member calls restore(spec, size, body): body is a reader over the member’s bytes and size its declared length; the member’s SHA-256 is VERIFIED as body is consumed (and any bytes restore leaves unread are drained through the hasher), so a corrupt/truncated member is caught before the next.

read_bundle_streaming_atomic

The v3-capable atomic single-pass reader — the inc2 superset of read_bundle_streaming_staged. Reads v1, v2 AND v3 bundles in one forward pass, dispatching each member to restorer by its MemberFraming: a Sidecar member is staged to a bounded temp under stage_dir and size/SHA-VERIFIED BEFORE restore_sidecar (verify-then-apply, unchanged from v2); a ChunkedTrailer member streams chunk-by-chunk through begin_chunked / write_chunk (the reader hashing + counting, NEVER staging the whole member) and is committed by finish_chunked ONLY when the TRAILING trailer matched — so a corrupt/truncated chunked member applies nothing (atomic). on_head(data_only, source) fires once after the MANIFEST, before any member. Enforces the v2/v3 declared order + per-member framing + the v1 completeness, exactly as [read_bundle_core].

read_bundle_streaming_staged

VERIFY-THEN-APPLY single-pass reader: like read_bundle_streaming_with_head but each member is staged to a bounded temp under stage_dir and its size + SHA-256 are VERIFIED from that temp BEFORE restore(spec, &verified_temp) is called (and the temp is freed after). This closes the apply-then-verify window — a corrupt/tampered member is rejected before it is applied — while keeping staging bounded to the largest single member. Reads v1 and v2 bundles. NOTE: this is the inc1 staged reader — it verifies-then-applies but does NOT own per-member rollback, so an import aborted mid-stream may have applied EARLIER members. inc2’s read_bundle_streaming_atomic adds the atomic shadow-restore (a FULL managed-DB member is swapped in only after its trailing SHA verifies, so a bad DB member applies NOTHING to live — per-member all-or-nothing); even there, the bundle is not whole-bundle atomic (the cross-member window) and a --data-only DB member keeps this non-atomic sidecar path.

read_bundle_streaming_with_head

Like read_bundle_streaming, but also surfaces the bundle HEAD — the data_only flag and the SourceInfo — via on_head, called ONCE after the MANIFEST is parsed and validated, BEFORE any member is handed to restore. The import driver needs both ahead of the first member (to re-key each member to the TARGET project, and to branch the data-only vs full restore), and the v2 MANIFEST (like the v1 one) carries them before any member — so a single forward pass can learn them up front. Reads both v2 and v1 bundles, exactly as read_bundle_streaming does.

read_manifest

Read + validate the MANIFEST (the first tar entry) from bundle, without extracting any member.

write_bundle

Write a bundle at out from members already staged on disk: compute each member’s size + SHA-256, build + validate the MANIFEST, then write the tar with the MANIFEST first and every member streamed from its staging file (bounded memory). Returns the MANIFEST that was written.

write_bundle_streaming

Stream a v2 bundle to out, producing each member JUST BEFORE it is appended and freeing its staging temp JUST AFTER — so the transient on-disk staging never exceeds the largest single member (O(largest member), never O(whole bundle)). For each MemberSpec in specs (restore order), stage(spec) stages THAT member to a temp and returns its path; the member is appended (sidecar + data) and its temp is then deleted before the next is staged.

write_bundle_streaming_v3

Stream a v3-capable bundle to out, dispatching each member by its MemberFraming — the chunked-aware superset of write_bundle_streaming the export driver uses. For a Sidecar member, stage(spec) stages it to a bounded temp (returned path), which is appended then deleted (the inc1 O(largest-member) footprint, unchanged); for a ChunkedTrailer member, open(spec) yields a Box<dyn Read> over the member’s bytes (an engine dump stream) that is piped STRAIGHT through append_member_chunked in chunk_size pieces — NO temp, O(chunk) disk + memory. The MANIFEST is v3 iff any member is chunked (else v2 — see BundleStreamWriter::new).