appcore-sync

Release candidate appcore-sync 1.0.2-rc.2. Select this RC explicitly; it does not replace earlier stable releases or certify a deployment. Internal AppCore dependencies, where present, are pinned to each dependency’s independent RC.

English guide | Guia em português | Guide français

Conservative leader-to-follower replication with versioned wire, log, snapshots, checkpoints, outbox, receiver and transport contracts.

Responsibility: conservative leader-to-follower replication and local durability helpers.

Internal dependencies: appcore-core, appcore-distributed-contracts, appcore-ops, appcore-transport.

Main API: node role/status/peer/heartbeat and SyncMessage; V1 wire codec; replication logs/snapshots; memory/file checkpoints and outbox; receiver state/acknowledgement; follower client; HTTP transport; peer discovery; retry, metrics and SyncError.

Identity, protocol, sequence and hash-chain validation are mandatory. This crate is not RAFT, multi-master consensus or a domain conflict resolver. Reexported opaque-envelope contracts expose the 1,024-byte MAX_OPAQUE_MESSAGE_ID_BYTES retention bound.

SyncConflict is a bounded, payload-free UI contract containing peer, sequence, local/remote SHA-256 and a typed reason. InMemorySyncConflictStore records conflicts and applies SyncConflictResolutionRequest idempotently; replaying the same key returns a receipt, while a different resolution cannot overwrite the first decision. The store records decisions only; it does not merge domain payloads or bypass leader-to-follower rules.

File logs, snapshots, checkpoints and outbox records are versioned and bounded. The receiver validates the complete incoming batch, sequence range and record sizes before mutating the replication log or checkpoint.

FileSyncCheckpointStore scans V1 through a fixed 16 KiB reader. Startup validation retains no peer map, and lookup validates the complete file while owning only the requested hash. Mutation still builds the canonical sorted map once, but writes it directly through a fixed buffer instead of keeping an additional file-sized String. The public ceilings are 8 MiB, 65,536 non-empty records and 256 UTF-8 bytes per peer ID; an encoded line is bounded before it can grow the scratch buffer. Duplicate peers keep the last value as before and the next successful mutation canonicalizes them.

FileReplicationLog scans V1 incrementally and retains only a compact sorted vector of sequence/record-index pairs plus per-record offsets, lengths and digests, not every payload. The in-memory implementation uses the same flat sorted index and binary-search lookup, avoiding hash-bucket overhead for bounded local logs. Concurrent instances validate the hash-chain anchor and scan only a new tail; atomic snapshot replacement is treated as a new generation and rebuilt safely. events_page limits reads to 1,024 records and 48 MiB before allocation. The complete events_since remains source-compatible but rejects a file read above those limits. Deployment-owned sync tools should use pages; no Runtime sync CLI is shipped. The log remains capped at 256 MiB, each payload at 1 MiB and the index at 262,144 records. Runtime HTTP batches additionally stop at 1 MiB of raw events; the encoded V1 JSON envelope is capped at 5 MiB so even the worst-case numeric byte representation of one valid payload remains transferable. The V1 encoder serializes borrowed identity, message and event fields directly into its required output string. It does not clone the complete batch before encoding, and its JSON remains byte-identical to the owned V1 contract.

ReplicationSnapshot::try_from_records consumes owned sequence/payload pairs and moves their allocations into a checksum-protected V1 snapshot. ReplicationSnapshot::validate checks the same format, count, payload, sequence and checksum invariants through a shared reference, without creating a second payload collection. Persistent providers can therefore validate before mutation while retaining one semantic snapshot owner. Memory-backed consumers that own the snapshot can call InMemoryReplicationLog::restore_snapshot_owned to validate and move payloads directly into the log, avoiding simultaneous snapshot and destination copies.

In 1.0.2-rc, ReplicationLog::len, last_index and is_empty return SyncResult. Persistent providers surface observation failures instead of substituting zero or stale state. Consumers must handle the result before updating; see release/fallible-replication-log-observations.md.

The 1.0.2-rc FileSyncOutbox uses the explicit appcore-sync-outbox-v2 append-only binary journal. Enqueue and acknowledgement sync one integrity-chained frame; readers scan only a new tail, and bounded compaction atomically retains pending messages. Only an incomplete final frame is recoverable. A complete corrupt, V1, unversioned or future-format file fails closed. Drain the V1 queue before upgrading and the V2 queue before rollback; see release/outbox-v2-migration.md.

The resident file-outbox state contains only batch IDs, journal offsets, encoded lengths, payload digests and retry metadata. Enqueue measures JSON in a bounded pass and then writes it through a fixed 64 KiB buffer; it does not keep an encoded copy. Front and page reads decode and verify one indexed message at a time. Each batch ID is shared with transactional tail-scan state, so refresh clones only handles instead of copying every pending identifier. The source-compatible messages() snapshot still returns an owned Vec, so memory-sensitive consumers must use bounded pages.

InMemorySyncOutbox also obtains the exact encoded length with an overflow-checked JSON counter rather than allocating and discarding a complete encoded message. A valid 4 MiB batch measured 10.92 ms p50 and 17.52 MiB peak RSS on Apple M1, down from 23.55 ms and 45.73 MiB with the temporary buffer.

The receiver's 10,000-entry processed-batch window retains one shared allocation per batch_id across duplicate lookup and acceptance-order eviction. Applying 10,000 batches with 128-byte IDs measured 58.27 ms p50 and 15.27 MiB peak RSS on Apple M1, down from 61.29 ms and 17.45 MiB with duplicate strings. Receiver and outbox boundaries reject empty IDs, control characters and IDs above 1,024 UTF-8 bytes before retaining them. This keeps the fixed entry window bounded by bytes as well as count. Duplicate rejection and oldest-first eviction are unchanged.

The additive 1.0.2-rc SyncOutbox paging contract exposes peek, stats, mark_attempt, next_ready and ordered partial receipts. Page reads are capped at 1,024 messages and 48 MiB before payload clones. The in-memory provider and file providers implement exact paging and retry observations. File attempts and ordered receipts are hash-chained journal frames that survive restart. A provider can call encoded_sync_message_bytes to obtain the exact compact JSON size without an encoded copy, then call write_sync_message_json to stream that same canonical representation to a bounded writer. String escaping remains Serde-compatible and event bytes are emitted through a fixed 16 KiB scratch buffer. A receipt is measured and serialized directly through the fixed 64 KiB writer; the maximum escaped-ID fixture no longer materializes its 2,086,913-byte JSON buffer in production. Journal scans borrow unescaped IDs from the frame. An external provider relying on the compatibility defaults still compiles: it returns at most the front message, reports unknown extended statistics and rejects persisted attempts or multi-message receipts explicitly.

FollowerSyncClient uses this bounded contract directly. Each failed transport call records retry readiness, success applies an exact receipt, and draining exposes the last acknowledged batch for checkpoint progress. The complete pending_messages snapshot remains for source compatibility; new consumers should use pending_page and outbox_stats.

HttpSyncTransport owns a reusable bounded HTTP client. with_timeout_ms keeps the uniform V1 deadline, while with_timeouts selects independent connect/admission, read and write deadlines.

cargo test -p appcore-sync

The default ReplicationLog::events_page is a full-read adapter for external providers: it validates limits, calls events_since, then moves selected payloads into the bounded result. It does not bound that initial read. Providers must override paging to enforce record/byte limits before reads or clones; internal memory/file providers already do so. A smaller returned page alone is not proof of bounded materialization. The consumer regression test is cargo test -p appcore-sync --test external_log_paging.

Maturity: stable conservative RC profile with strict V1 decoding.

Stable documentation

Stable ID: ACR-012. See the supplemental architecture and integration guide. This permanent ID remains valid if the wiki page moves.

For opaque payloads larger than one request, split_sync_payload creates bounded chunks with per-chunk and full-payload SHA-256 digests. SyncChunkAssembler accepts out-of-order chunks, reports missing ranges without payload, and treats identical replay as idempotent. Invalid ranges, overlaps and digest mismatches fail closed; the V1 wire message is unchanged.

Application envelope migration.

Opaque revision causality

This section describes unpublished source changes. Verify the exact registry package before consumer adoption.

ObjectRevisionV1, RevisionChainV1 and decide_object_revision_v1 classify already_applied, ancestor_echo, applicable, concurrent_branch and snapshot_required. The chain contains at most 1,024 revisions with 128-byte identity fields. Authenticate every event and bind its tenant before constructing evidence: chain validation is structural, not cryptographic. Missing links, cycles, object mismatch and reused event/revision identities fail closed. Equality of content or timestamps never establishes ancestry. An intentional revert uses a fresh revision/event. Advance the transport checkpoint independently when accepting an ancestor echo; call validate_checkpoint_head_v1 before mutating metadata.

Retransmission and checkpoint serialization

Unpublished source fix: an identical authenticated replay returns an accepted ACK with zero new records; divergent reuse of a retained batch_id fails with divergent batch_id reuse. The receiver retains at most 10,000 ID/digest pairs. This window is process-local, not permanent replay evidence; after restart or eviction the existing old-sequence checkpoint policy applies. Authenticate and isolate the source before calling the receiver. Distinct receiver states must share the same replication-log handle when they share checkpoints.

Checkpoint observation, overlap verification, log append and sequence/hash publication share the log lock. Concurrent batches cannot publish a stale smaller checkpoint or combine one batch’s sequence with another batch’s hash. Provider errors can still leave an appended log with an older checkpoint; an identical retry verifies overlap before publishing the checkpoint. The sender removes its outbox receipt only after transport acknowledgement. Losing the first ACK therefore causes a safe retransmission and eventual drain. External processes and independent log handles require provider/deployment transaction coordination.

Release candidate — 2026-10-14

2.0.0-alpha.5: source changes.

Canonical service history (unpublished source)

sync::canonical::FileCanonicalHistoryV1 preserves original SyncMessage metadata and event bytes, with transferable CanonicalBatchV1 epoch/writer evidence. CanonicalStreamV1 identifies tenant/cluster/service independently of the current Core or address. CanonicalHeadV1 binds sequence, original batch hash and a second digest over stream, epoch, writer and history. Existing V1 receiver, wire, log and snapshot formats are unchanged; there is no automatic import or conversion.

Use explicit create only for initial provisioning; restart uses open_existing. Every operation reloads and validates durable state under an OS try-lock. Writes sync an exclusive temporary file, atomically rename it and sync the directory where supported. Lost ACKs accept identical persisted records after restart; gaps, divergent IDs/overlaps, backwards epochs and changed writers within one epoch fail closed. Corruption or missing storage never creates a new history. A failed persistence transaction must be reopened and its result inspected: rename may already have succeeded.

batches(offset, limit) returns up to 64 complete lineage records; authenticate the entire record before replicate_authenticated. This copies canonical history only, not application materialization or effect transactions. Historical epoch evidence is not a current lease. The host must authenticate tenant/service/source and historical authority before admitting records; hashes are not signatures.

with_verified_head(required, action) compares the complete durable head and holds the history lock during the callback. The required head must come from a trusted durability policy, not the candidate itself. An empty or stale replica, even at the same sequence with a different hash, cannot run the action. Inside the callback compose a current exact authority fence around writer commit, using a consistent history-then-authority lock order. The co-located reference is FileControlPlane::with_fenced_service_write; do not reenter either provider. A remote ServiceLeaseAuthority::check is an observation, not an atomic remote commit. The primitive does not promote a node or issue a readiness certificate by itself.

Discover the writer through ServiceLeaseAuthority::inspect for the same tenant/cluster/service, then resolve its Core through deployment discovery and authenticate the endpoint. Inspection may return an expired grant. A former writer rejoins as a replica; divergent history is rejected for explicit operator/application snapshot reconciliation. Do not reset sequence or rename the source to hide divergence. An external rollback of both authority and history is outside this reference; certified storage custody remains mandatory.

Limits: 16 MiB encoded file, 1,024 batches, 16,384 events and 4 MiB total raw payload, each event at most 1 MiB. Identity fields use a 128-byte ASCII alphanumeric/-_. profile. This small reference reloads the whole bounded file; JSON decoding has allocation overhead before decoded admission, so it is not a large-log streaming provider. Run filesystem operations on bounded blocking workers. Capacity exhaustion stops writes; no evidence is evicted or compacted. Application snapshots, replica-confirmation policy and physical mesh qualification remain separate deployment work.

Each successful append commits immediately; returning an error from the callback does not roll back an earlier append. Reopen and inspect the durable head before retrying an uncertain result.

Canonical-copy durability (unpublished source)

DurabilityTargetV1 fixes the original batch ID, complete head, writer Core and epoch. DurabilityPolicyV1 explicitly selects local, master or master_and_replica. evaluate_durability_v1 returns DurabilityStatusV1 with confirmed, persisted_core_ids and missing_core_ids; pending never silently downgrades the policy. Master means the original writer for this immutable target, not whichever Core discovery currently returns. A retry keeps the same target and policy; pending/timeout does not roll back a write or authorize rerunning an application handler.

FileCanonicalHistoryV1::durability_target recovers an exact retained batch boundary. durability_receipt(target, local_core_id) reloads and validates retained bytes under the history lock, verifies all target fields, syncs the file and directory where supported, then issues a deterministic V1 receipt. Bind local_core_id to deployment-owned storage identity, never a requester field. An older target remains confirmable only as an exact retained ancestor, allowing lost-reply recovery after restart and later batches without duplicate append. No separate receipt journal or history-format change is needed. Genesis, partial batch boundaries, missing/corrupt/busy storage and divergent targets cannot issue a receipt.

Authenticate the entire receipt and bind persisted_by_core_id to the authenticated sender before evaluation. JSON decoding and digests are not authentication. decode_durability_receipt_v1 bounds input to 16 KiB and rejects unknown/unversioned input with NO MORE SUPPORTED PLEASE UPDATE; evaluation admits at most 16 receipts. Repeated evidence from one Core is idempotent and never counts twice. Every receipt must match the complete target; unexpected Cores cannot satisfy selected requirements. Receipt fields are not a current lease, availability probe, application materialization proof or consensus certificate. Filesystem sync remains subject to the platform/filesystem's certified durability guarantees.

The deployment persists the selected policy, trusted target and authenticated confirmations in its recovery boundary before reporting success or retiring the canonical batch. It must serialize outbox mutations and bind retirement to the same batch content, not merely an ID check racing another consumer. These primitives retain no unbounded pending-operation map and do not automatically mutate an outbox. The existing FollowerSyncClient keeps its transport-ACK contract; applications requiring replica durability must explicitly compose this confirmation path. The integration test retains a real file outbox while pending, restarts it after a lost replica reply and retires only after both matching receipts arrive.

These policies confirm canonical history copies, not a pre-canonical local application transaction/outbox. Existing local-first offline application durability remains application-owned. Do not label a mere local outbox entry as master/replica confirmation. A trusted confirmed target can supply the required head for promotion checks only when the deployment retains it and authenticates its evidence; current exact leadership and atomic write fencing remain mandatory. No production integration, hardware power-loss qualification or zero-loss guarantee is asserted.

Negotiated HA availability — 1.0.2-rc.2

The opt-in peer.availability.v1 profile separates local transport failure, authenticated rejection, pending execution and durable confirmation. It preserves frozen V1 behavior for V1 peers and never treats a transport ACK as business durability. Deployment composition must supply the durable executor and distinct request/response authentication. See the availability contract and adoption requirements.