Expand description
WAL checksum primitives, integrity helpers, and native commit protocol.
Fault Matrix (bd-966ja) for batched append and publish paths:
This matrix covers the byte-oriented append helpers in wal::WalFile and
the current durable-publish helper group_commit::write_consolidated_frames.
It intentionally stops at WAL durability. Later SHM / snapshot-plane publish
remains the caller’s responsibility (see native_commit).
| Stage | Primary APIs | Fault conditions | Surface | State after fault |
|---|---|---|---|---|
| Batch shape + serialization | prepare_frame_bytes_with_transforms_into | Frame-count mismatch, page-size mismatch, frame_count * frame_size overflow, salt/checksum helper failure while constructing transforms | WalCorrupt, DatabaseFull, or helper error | WAL file and in-memory WAL counters stay unchanged; caller-owned scratch may contain partial serialized bytes |
| Append-window validation | prepared_append_window_still_current | File size changed, generation header changed, short header read, header parse failure | Ok(false) for stale window; WalCorrupt / parse error for malformed WAL | No append occurs; caller must rebuild from the new seed or treat the WAL as corrupt |
| Checksum finalization | finalize_prepared_frame_bytes | Prepared buffer length mismatch, frame_count * frame_size overflow, salt/checksum write failure | WalCorrupt, DatabaseFull, or checksum helper error | No on-disk mutation; caller buffer may have partially rewritten checksum fields |
| Durable byte append | append_finalized_prepared_frame_bytes | Frame-count overflow, prepared buffer length mismatch, VFS write failure, post-write state advance overflow guard | DatabaseFull, WalCorrupt, or VFS error | Pre-write validation faults leave WAL state unchanged; a write failure happens before advance_state_after_write, so in-memory counters do not advance even though the on-disk tail may need replay/validation |
| One-shot prepared append | append_prepared_frame_bytes | Any finalization fault plus any finalized-append fault | Propagated error from the lower stage | Same guarantees as the composed lower layers: no publish on finalize failure; write-path faults may leave an untrusted tail that recovery must trim or validate |
| Fused batched append | append_frames | Test/fault-injection busy hook, page-size mismatch in any frame, batch-size overflow, salt/checksum helper failure, finalized append failure, test-only after-append injected fault | Busy, WalCorrupt, DatabaseFull, helper error, or injected test fault | Assembly faults leave frame_scratch restored and counters unchanged; an after-append injected fault is special because bytes/state may already be advanced even though the function returns Err |
| Durable publish | write_consolidated_frames, direct WalFile::sync callers | Frame-batch byte-size overflow, any append_frames fault, sync(FULL) failure after a successful append | Internal, append error, or VFS sync error | If sync fails, WAL bytes may already be appended and WalFile state may already reflect them, but durability/publish is not established, so higher layers must not advertise the batch as committed |
Re-exports§
pub use cell_delta_commit::CellDeltaDescriptor;pub use cell_delta_commit::FullPageFrame;pub use cell_delta_commit::MixedCommitStats;pub use cell_delta_commit::MixedFrameSubmission;pub use cell_delta_commit::build_cell_delta_frames;pub use cell_delta_commit::serialize_mixed_frames;pub use cell_delta_wal::CELL_DELTA_CHECKSUM_SIZE;pub use cell_delta_wal::CELL_DELTA_FRAME_MARKER;pub use cell_delta_wal::CELL_DELTA_HEADER_SIZE;pub use cell_delta_wal::CELL_DELTA_MAX_DATA_SIZE;pub use cell_delta_wal::CELL_DELTA_MIN_FRAME_SIZE;pub use cell_delta_wal::CellDeltaWalFrame;pub use cell_delta_wal::CellOp;pub use cell_delta_wal::WalRecoverySummary;pub use cell_delta_wal::extract_page_number_from_marker;pub use cell_delta_wal::is_cell_delta_frame;pub use checkpoint::CheckpointMode;pub use checkpoint::CheckpointPlan;pub use checkpoint::CheckpointPostAction;pub use checkpoint::CheckpointProgress;pub use checkpoint::CheckpointState;pub use checkpoint::plan_checkpoint;pub use checkpoint_executor::CheckpointExecutionResult;pub use checkpoint_executor::CheckpointTarget;pub use checkpoint_executor::execute_checkpoint;pub use checksum::BTREE_PAGE_TYPE_FLAGS;pub use checksum::CRASH_MODEL_SECTOR_SIZES;pub use checksum::ChecksumFailureKind;pub use checksum::CrashModelContract;pub use checksum::HashTier;pub use checksum::IntegrityCheckIssue;pub use checksum::IntegrityCheckLevel;pub use checksum::IntegrityCheckReport;pub use checksum::PAGE_CHECKSUM_RESERVED_BYTES;pub use checksum::RecoveryAction;pub use checksum::SQLITE_DB_HEADER_RESERVED_OFFSET;pub use checksum::SQLITE_DB_HEADER_SIZE;pub use checksum::SqliteWalChecksum;pub use checksum::WAL_FORMAT_VERSION;pub use checksum::WAL_FRAME_HEADER_SIZE;pub use checksum::WAL_HEADER_SIZE;pub use checksum::WAL_MAGIC_BE;pub use checksum::WAL_MAGIC_LE;pub use checksum::WalChainInvalidReason;pub use checksum::WalChainValidation;pub use checksum::WalFecRepairOutcome;pub use checksum::WalFrameHeader;pub use checksum::WalHeader;pub use checksum::WalRecoveryDecision;pub use checksum::WalSalts;pub use checksum::Xxh3Checksum128;pub use checksum::attempt_wal_fec_repair;pub use checksum::compute_wal_frame_checksum;pub use checksum::configure_page_checksum_reserved_bytes;pub use checksum::content_address_hash_128;pub use checksum::crash_model_contract;pub use checksum::crc32c_checksum;pub use checksum::detect_torn_write_in_wal;pub use checksum::integrity_check_database_header;pub use checksum::integrity_check_level1_page;pub use checksum::integrity_check_level2_btree;pub use checksum::integrity_check_level3_overflow_chain;pub use checksum::integrity_check_level4_cross_reference;pub use checksum::integrity_check_level5_schema;pub use checksum::integrity_check_sqlite_file_level1;pub use checksum::integrity_hash_xxh3_128;pub use checksum::is_valid_btree_page_type;pub use checksum::merge_integrity_reports;pub use checksum::page_checksum_reserved_bytes;pub use checksum::read_page_checksum;pub use checksum::read_wal_frame_checksum;pub use checksum::read_wal_frame_salts;pub use checksum::read_wal_header_checksum;pub use checksum::read_wal_header_salts;pub use checksum::recover_wal_frame_checksum_mismatch;pub use checksum::recovery_action_for_checksum_failure;pub use checksum::sqlite_wal_checksum;pub use checksum::supports_torn_write_sector_size;pub use checksum::tier_for_algorithm;pub use checksum::validate_wal_chain;pub use checksum::validate_wal_header_checksum;pub use checksum::verify_page_checksum;pub use checksum::verify_wal_fec_source_hash;pub use checksum::wal_fec_source_hash_xxh3_128;pub use checksum::wal_frame_db_size;pub use checksum::wal_header_checksum;pub use checksum::write_page_checksum;pub use checksum::write_wal_frame_checksum;pub use checksum::write_wal_frame_salts;pub use checksum::write_wal_header_checksum;pub use checksum::write_wal_header_salts;pub use checksum::zero_page_checksum_trailer;pub use group_commit::ConsolidationMetrics;pub use group_commit::ConsolidationMetricsSnapshot;pub use group_commit::ConsolidationPhase;pub use group_commit::FrameSubmission;pub use group_commit::GLOBAL_CONSOLIDATION_METRICS;pub use group_commit::GroupCommitConfig;pub use group_commit::GroupCommitConsolidator;pub use group_commit::PhaseHistogram;pub use group_commit::PhasePercentiles;pub use group_commit::SubmitOutcome;pub use group_commit::TransactionConflictSnapshot;pub use group_commit::TransactionFrameBatch;pub use group_commit::TransactionFrameBatchContext;pub use group_commit::WakeReasonCounters;pub use group_commit::WakeReasonSnapshot;pub use group_commit::commit_phase_timing_enabled;pub use group_commit::commit_phase_timing_forced_enabled;pub use group_commit::detailed_consolidation_metrics_enabled;pub use group_commit::set_commit_phase_timing_enabled;pub use group_commit::write_consolidated_frames;pub use metrics::GLOBAL_GROUP_COMMIT_METRICS;pub use metrics::GLOBAL_WAL_FEC_REPAIR_METRICS;pub use metrics::GLOBAL_WAL_METRICS;pub use metrics::GLOBAL_WAL_RECOVERY_METRICS;pub use metrics::GroupCommitMetrics;pub use metrics::GroupCommitMetricsSnapshot;pub use metrics::WalFecRepairCounters;pub use metrics::WalFecRepairCountersSnapshot;pub use metrics::WalMetrics;pub use metrics::WalMetricsSnapshot;pub use metrics::WalRecoveryCounters;pub use metrics::WalRecoveryCountersSnapshot;pub use parallel_wal::FsyncPolicy;pub use parallel_wal::PARALLEL_WAL_COMPATIBILITY_SELECTOR;pub use parallel_wal::PARALLEL_WAL_FLUSH_SCENARIO_ID;pub use parallel_wal::PARALLEL_WAL_LANE_POLICY_VERSION;pub use parallel_wal::PARALLEL_WAL_STAGE_SCENARIO_ID;pub use parallel_wal::ParallelWalBatch;pub use parallel_wal::ParallelWalCommitCertificate;pub use parallel_wal::ParallelWalConfig;pub use parallel_wal::ParallelWalControlSurface;pub use parallel_wal::ParallelWalCoordinator;pub use parallel_wal::ParallelWalDecisionAction;pub use parallel_wal::ParallelWalDecisionRecord;pub use parallel_wal::ParallelWalFallbackReason;pub use parallel_wal::ParallelWalFrame;pub use parallel_wal::ParallelWalLaneBatch;pub use parallel_wal::ParallelWalLaneStager;pub use parallel_wal::ParallelWalOperatingMode;pub use parallel_wal::ParallelWalOrderedResidue;pub use parallel_wal::ParallelWalShadowVerdict;pub use parallel_wal::ParallelWalTraceRecord;pub use parallel_wal::SegmentHeader;pub use parallel_wal::SegmentRecoveryOptions;pub use parallel_wal::SegmentRecoveryResult;pub use parallel_wal::cleanup_segments;pub use parallel_wal::default_parallel_wal_lane_count;pub use parallel_wal::delete_segment;pub use parallel_wal::list_segments;pub use parallel_wal::max_durable_epoch;pub use parallel_wal::parallel_wal_coordinator_for_path;pub use parallel_wal::parallel_wal_fallback_reason_name;pub use parallel_wal::parallel_wal_mode_name;pub use parallel_wal::parallel_wal_shadow_verdict_name;pub use parallel_wal::parallel_wal_should_shadow_compare;pub use parallel_wal::read_segment;pub use parallel_wal::recover_and_apply_segments;pub use parallel_wal::recover_segments;pub use parallel_wal::remove_parallel_wal_coordinator;pub use parallel_wal::resolve_parallel_wal_control_surface_from_env;pub use parallel_wal::segment_path;pub use parallel_wal::write_segment;pub use per_core_buffer::AppendOutcome;pub use per_core_buffer::BufferConfig;pub use per_core_buffer::BufferState;pub use per_core_buffer::DEFAULT_BUFFER_SLOT_COUNT;pub use per_core_buffer::EpochConfig;pub use per_core_buffer::EpochFlushBatch;pub use per_core_buffer::EpochOrderCoordinator;pub use per_core_buffer::FallbackDecision;pub use per_core_buffer::OverflowPolicy;pub use per_core_buffer::PerCoreWalBuffer;pub use per_core_buffer::PerCoreWalBufferPool;pub use per_core_buffer::WalRecord;pub use per_core_buffer::thread_buffer_slot;pub use recovery_fence::CheckpointChecksumVerdict;pub use recovery_fence::ExpectedPageChecksum;pub use recovery_fence::PidOwnedLockEntry;pub use recovery_fence::PidOwnedLockRegistry;pub use recovery_fence::RECOVERY_FENCE_BACKOFF;pub use recovery_fence::RECOVERY_FENCE_MAX_RETRIES;pub use recovery_fence::RecoveryFence;pub use recovery_fence::RecoveryFenceGuard;pub use recovery_fence::ensure_db_fsync_before_wal_truncate;pub use recovery_fence::execute_recovery_barrier;pub use recovery_fence::fsync_db_file_full;pub use recovery_fence::pid_alive_os;pub use recovery_fence::verify_checkpoint_checksum_prefix;pub use telemetry::NoOpWalObserver;pub use telemetry::WalTelemetryEvent;pub use telemetry::WalTelemetryObserver;pub use telemetry::WalTelemetryRingBuffer;pub use telemetry::WalTelemetrySnapshot;pub use telemetry::wal_telemetry_snapshot;pub use wal::WalFile;pub use wal::WalGenerationIdentity;pub use wal_fec::DEFAULT_RAPTORQ_REPAIR_SYMBOLS;pub use wal_fec::MAX_RAPTORQ_REPAIR_SYMBOLS;pub use wal_fec::WAL_FEC_GROUP_META_MAGIC;pub use wal_fec::WAL_FEC_GROUP_META_VERSION;pub use wal_fec::WalFecDecodeProof;pub use wal_fec::WalFecGroupId;pub use wal_fec::WalFecGroupMeta;pub use wal_fec::WalFecGroupMetaInit;pub use wal_fec::WalFecGroupRecord;pub use wal_fec::WalFecRecoveredGroup;pub use wal_fec::WalFecRecoveryConfig;pub use wal_fec::WalFecRecoveryFallbackReason;pub use wal_fec::WalFecRecoveryLog;pub use wal_fec::WalFecRecoveryOutcome;pub use wal_fec::WalFecRepairEvent;pub use wal_fec::WalFecRepairEvidenceCard;pub use wal_fec::WalFecRepairEvidenceQuery;pub use wal_fec::WalFecRepairMetricsSnapshot;pub use wal_fec::WalFecRepairPipeline;pub use wal_fec::WalFecRepairPipelineConfig;pub use wal_fec::WalFecRepairPipelineStats;pub use wal_fec::WalFecRepairSeverityBucket;pub use wal_fec::WalFecRepairSeverityHistogram;pub use wal_fec::WalFecRepairSource;pub use wal_fec::WalFecRepairWitnessTriple;pub use wal_fec::WalFecRepairWorkItem;pub use wal_fec::WalFecScanResult;pub use wal_fec::WalFrameCandidate;pub use wal_fec::append_wal_fec_group;pub use wal_fec::build_source_page_hashes;pub use wal_fec::ensure_wal_with_fec_sidecar;pub use wal_fec::find_wal_fec_group;pub use wal_fec::generate_wal_fec_repair_symbols;pub use wal_fec::identify_damaged_commit_group;pub use wal_fec::persist_wal_fec_raptorq_repair_symbols;pub use wal_fec::query_raptorq_repair_evidence;pub use wal_fec::raptorq_repair_events_snapshot;pub use wal_fec::raptorq_repair_evidence_snapshot;pub use wal_fec::raptorq_repair_metrics_snapshot;pub use wal_fec::read_wal_fec_raptorq_repair_symbols;pub use wal_fec::record_raptorq_recovery_log;pub use wal_fec::recover_wal_fec_group_with_config;pub use wal_fec::recover_wal_fec_group_with_decoder;pub use wal_fec::recovery_log_from_outcome;pub use wal_fec::reset_raptorq_repair_telemetry;pub use wal_fec::scan_wal_fec;pub use wal_fec::wal_fec_path_for_wal;pub use wal_index::WAL_CKPT_INFO_BYTES;pub use wal_index::WAL_CKPT_LOCK;pub use wal_index::WAL_INDEX_HASH_MASK;pub use wal_index::WAL_INDEX_HASH_MULTIPLIER;pub use wal_index::WAL_INDEX_HASH_SLOTS;pub use wal_index::WAL_INDEX_HDR_BYTES;pub use wal_index::WAL_INDEX_PAGE_ARRAY_ENTRIES;pub use wal_index::WAL_INDEX_VERSION;pub use wal_index::WAL_LOCK_SLOT_COUNT;pub use wal_index::WAL_READ_LOCK_BASE;pub use wal_index::WAL_READ_MARK_COUNT;pub use wal_index::WAL_RECOVER_LOCK;pub use wal_index::WAL_SHM_FIRST_HEADER_BYTES;pub use wal_index::WAL_SHM_FIRST_HEADER_U32_SLOTS;pub use wal_index::WAL_SHM_FIRST_USABLE_PAGE_ENTRIES;pub use wal_index::WAL_SHM_HASH_BYTES;pub use wal_index::WAL_SHM_PAGE_ARRAY_BYTES;pub use wal_index::WAL_SHM_SEGMENT_BYTES;pub use wal_index::WAL_SHM_SUBSEQUENT_USABLE_PAGE_ENTRIES;pub use wal_index::WAL_WRITE_LOCK;pub use wal_index::WalCkptInfo;pub use wal_index::WalHashLookup;pub use wal_index::WalIndexHashSegment;pub use wal_index::WalIndexHdr;pub use wal_index::WalIndexSegmentKind;pub use wal_index::decode_native_u32;pub use wal_index::encode_native_u32;pub use wal_index::parse_shm_header;pub use wal_index::simple_modulo_slot;pub use wal_index::usable_page_entries;pub use wal_index::wal_index_hash_slot;pub use wal_index::wal_index_hdr_copies_match;pub use wal_index::write_shm_header;
Modules§
- cell_
delta_ commit - Cell-Delta WAL Commit Integration (C4: bd-l9k8e.4)
- cell_
delta_ wal - Cell-Delta WAL Frame Format (C4-WAL: bd-l9k8e.10)
- checkpoint
- WAL checkpoint planning primitives for PASSIVE/FULL/RESTART/TRUNCATE modes.
- checkpoint_
executor - WAL checkpoint execution engine.
- checksum
- WAL checksum and integrity helpers.
- group_
commit - Group commit with consolidation for WAL frame writes (bd-ncivz.3).
- metrics
- WAL observability metrics.
- native_
commit - Native mode commit protocol (§7.11).
- parallel_
wal - Parallel WAL coordinator (D1: bd-3wop3.1).
- per_
core_ buffer - recovery_
compaction - Native mode recovery algorithm (§7.12) and ECS storage reclamation (§7.13).
- recovery_
fence - Recovery fencing, dead-PID lock release, and checkpoint checksum validation
primitives for
bd-yfdb6(OPS-3-2 CRITICAL). - telemetry
- Machine-validated WAL telemetry schema.
- wal
- Core WAL file I/O layer.
- wal_fec
- WAL-FEC sidecar format (
.wal-fec) for self-healing WAL durability (§3.4.1). - wal_
index - WAL-index hash table primitives.