Skip to main content

Crate fsqlite_wal

Crate fsqlite_wal 

Source
Expand description

WAL checksum primitives, integrity helpers, and native commit protocol.

Fault Matrix (bd-966ja) for batched append and publish paths:

This matrix covers the byte-oriented append helpers in wal::WalFile and the current durable-publish helper group_commit::write_consolidated_frames. It intentionally stops at WAL durability. Later SHM / snapshot-plane publish remains the caller’s responsibility (see native_commit).

StagePrimary APIsFault conditionsSurfaceState after fault
Batch shape + serializationprepare_frame_bytes_with_transforms_intoFrame-count mismatch, page-size mismatch, frame_count * frame_size overflow, salt/checksum helper failure while constructing transformsWalCorrupt, DatabaseFull, or helper errorWAL file and in-memory WAL counters stay unchanged; caller-owned scratch may contain partial serialized bytes
Append-window validationprepared_append_window_still_currentFile size changed, generation header changed, short header read, header parse failureOk(false) for stale window; WalCorrupt / parse error for malformed WALNo append occurs; caller must rebuild from the new seed or treat the WAL as corrupt
Checksum finalizationfinalize_prepared_frame_bytesPrepared buffer length mismatch, frame_count * frame_size overflow, salt/checksum write failureWalCorrupt, DatabaseFull, or checksum helper errorNo on-disk mutation; caller buffer may have partially rewritten checksum fields
Durable byte appendappend_finalized_prepared_frame_bytesFrame-count overflow, prepared buffer length mismatch, VFS write failure, post-write state advance overflow guardDatabaseFull, WalCorrupt, or VFS errorPre-write validation faults leave WAL state unchanged; a write failure happens before advance_state_after_write, so in-memory counters do not advance even though the on-disk tail may need replay/validation
One-shot prepared appendappend_prepared_frame_bytesAny finalization fault plus any finalized-append faultPropagated error from the lower stageSame guarantees as the composed lower layers: no publish on finalize failure; write-path faults may leave an untrusted tail that recovery must trim or validate
Fused batched appendappend_framesTest/fault-injection busy hook, page-size mismatch in any frame, batch-size overflow, salt/checksum helper failure, finalized append failure, test-only after-append injected faultBusy, WalCorrupt, DatabaseFull, helper error, or injected test faultAssembly faults leave frame_scratch restored and counters unchanged; an after-append injected fault is special because bytes/state may already be advanced even though the function returns Err
Durable publishwrite_consolidated_frames, direct WalFile::sync callersFrame-batch byte-size overflow, any append_frames fault, sync(FULL) failure after a successful appendInternal, append error, or VFS sync errorIf sync fails, WAL bytes may already be appended and WalFile state may already reflect them, but durability/publish is not established, so higher layers must not advertise the batch as committed

Re-exports§

pub use cell_delta_commit::CellDeltaDescriptor;
pub use cell_delta_commit::FullPageFrame;
pub use cell_delta_commit::MixedCommitStats;
pub use cell_delta_commit::MixedFrameSubmission;
pub use cell_delta_commit::build_cell_delta_frames;
pub use cell_delta_commit::serialize_mixed_frames;
pub use cell_delta_wal::CELL_DELTA_CHECKSUM_SIZE;
pub use cell_delta_wal::CELL_DELTA_FRAME_MARKER;
pub use cell_delta_wal::CELL_DELTA_HEADER_SIZE;
pub use cell_delta_wal::CELL_DELTA_MAX_DATA_SIZE;
pub use cell_delta_wal::CELL_DELTA_MIN_FRAME_SIZE;
pub use cell_delta_wal::CellDeltaWalFrame;
pub use cell_delta_wal::CellOp;
pub use cell_delta_wal::WalRecoverySummary;
pub use cell_delta_wal::extract_page_number_from_marker;
pub use cell_delta_wal::is_cell_delta_frame;
pub use checkpoint::CheckpointMode;
pub use checkpoint::CheckpointPlan;
pub use checkpoint::CheckpointPostAction;
pub use checkpoint::CheckpointProgress;
pub use checkpoint::CheckpointState;
pub use checkpoint::plan_checkpoint;
pub use checkpoint_executor::CheckpointExecutionResult;
pub use checkpoint_executor::CheckpointTarget;
pub use checkpoint_executor::execute_checkpoint;
pub use checksum::BTREE_PAGE_TYPE_FLAGS;
pub use checksum::CRASH_MODEL_SECTOR_SIZES;
pub use checksum::ChecksumFailureKind;
pub use checksum::CrashModelContract;
pub use checksum::HashTier;
pub use checksum::IntegrityCheckIssue;
pub use checksum::IntegrityCheckLevel;
pub use checksum::IntegrityCheckReport;
pub use checksum::PAGE_CHECKSUM_RESERVED_BYTES;
pub use checksum::RecoveryAction;
pub use checksum::SQLITE_DB_HEADER_RESERVED_OFFSET;
pub use checksum::SQLITE_DB_HEADER_SIZE;
pub use checksum::SqliteWalChecksum;
pub use checksum::WAL_FORMAT_VERSION;
pub use checksum::WAL_FRAME_HEADER_SIZE;
pub use checksum::WAL_HEADER_SIZE;
pub use checksum::WAL_MAGIC_BE;
pub use checksum::WAL_MAGIC_LE;
pub use checksum::WalChainInvalidReason;
pub use checksum::WalChainValidation;
pub use checksum::WalFecRepairOutcome;
pub use checksum::WalFrameHeader;
pub use checksum::WalHeader;
pub use checksum::WalRecoveryDecision;
pub use checksum::WalSalts;
pub use checksum::Xxh3Checksum128;
pub use checksum::attempt_wal_fec_repair;
pub use checksum::compute_wal_frame_checksum;
pub use checksum::configure_page_checksum_reserved_bytes;
pub use checksum::content_address_hash_128;
pub use checksum::crash_model_contract;
pub use checksum::crc32c_checksum;
pub use checksum::detect_torn_write_in_wal;
pub use checksum::integrity_check_database_header;
pub use checksum::integrity_check_level1_page;
pub use checksum::integrity_check_level2_btree;
pub use checksum::integrity_check_level3_overflow_chain;
pub use checksum::integrity_check_level4_cross_reference;
pub use checksum::integrity_check_level5_schema;
pub use checksum::integrity_check_sqlite_file_level1;
pub use checksum::integrity_hash_xxh3_128;
pub use checksum::is_valid_btree_page_type;
pub use checksum::merge_integrity_reports;
pub use checksum::page_checksum_reserved_bytes;
pub use checksum::read_page_checksum;
pub use checksum::read_wal_frame_checksum;
pub use checksum::read_wal_frame_salts;
pub use checksum::read_wal_header_checksum;
pub use checksum::read_wal_header_salts;
pub use checksum::recover_wal_frame_checksum_mismatch;
pub use checksum::recovery_action_for_checksum_failure;
pub use checksum::sqlite_wal_checksum;
pub use checksum::supports_torn_write_sector_size;
pub use checksum::tier_for_algorithm;
pub use checksum::validate_wal_chain;
pub use checksum::validate_wal_header_checksum;
pub use checksum::verify_page_checksum;
pub use checksum::verify_wal_fec_source_hash;
pub use checksum::wal_fec_source_hash_xxh3_128;
pub use checksum::wal_frame_db_size;
pub use checksum::wal_header_checksum;
pub use checksum::write_page_checksum;
pub use checksum::write_wal_frame_checksum;
pub use checksum::write_wal_frame_salts;
pub use checksum::write_wal_header_checksum;
pub use checksum::write_wal_header_salts;
pub use checksum::zero_page_checksum_trailer;
pub use group_commit::ConsolidationMetrics;
pub use group_commit::ConsolidationMetricsSnapshot;
pub use group_commit::ConsolidationPhase;
pub use group_commit::FrameSubmission;
pub use group_commit::GLOBAL_CONSOLIDATION_METRICS;
pub use group_commit::GroupCommitConfig;
pub use group_commit::GroupCommitConsolidator;
pub use group_commit::PhaseHistogram;
pub use group_commit::PhasePercentiles;
pub use group_commit::SubmitOutcome;
pub use group_commit::TransactionConflictSnapshot;
pub use group_commit::TransactionFrameBatch;
pub use group_commit::TransactionFrameBatchContext;
pub use group_commit::WakeReasonCounters;
pub use group_commit::WakeReasonSnapshot;
pub use group_commit::commit_phase_timing_enabled;
pub use group_commit::commit_phase_timing_forced_enabled;
pub use group_commit::detailed_consolidation_metrics_enabled;
pub use group_commit::set_commit_phase_timing_enabled;
pub use group_commit::write_consolidated_frames;
pub use metrics::GLOBAL_GROUP_COMMIT_METRICS;
pub use metrics::GLOBAL_WAL_FEC_REPAIR_METRICS;
pub use metrics::GLOBAL_WAL_METRICS;
pub use metrics::GLOBAL_WAL_RECOVERY_METRICS;
pub use metrics::GroupCommitMetrics;
pub use metrics::GroupCommitMetricsSnapshot;
pub use metrics::WalFecRepairCounters;
pub use metrics::WalFecRepairCountersSnapshot;
pub use metrics::WalMetrics;
pub use metrics::WalMetricsSnapshot;
pub use metrics::WalRecoveryCounters;
pub use metrics::WalRecoveryCountersSnapshot;
pub use parallel_wal::FsyncPolicy;
pub use parallel_wal::PARALLEL_WAL_COMPATIBILITY_SELECTOR;
pub use parallel_wal::PARALLEL_WAL_FLUSH_SCENARIO_ID;
pub use parallel_wal::PARALLEL_WAL_LANE_POLICY_VERSION;
pub use parallel_wal::PARALLEL_WAL_STAGE_SCENARIO_ID;
pub use parallel_wal::ParallelWalBatch;
pub use parallel_wal::ParallelWalCommitCertificate;
pub use parallel_wal::ParallelWalConfig;
pub use parallel_wal::ParallelWalControlSurface;
pub use parallel_wal::ParallelWalCoordinator;
pub use parallel_wal::ParallelWalDecisionAction;
pub use parallel_wal::ParallelWalDecisionRecord;
pub use parallel_wal::ParallelWalFallbackReason;
pub use parallel_wal::ParallelWalFrame;
pub use parallel_wal::ParallelWalLaneBatch;
pub use parallel_wal::ParallelWalLaneStager;
pub use parallel_wal::ParallelWalOperatingMode;
pub use parallel_wal::ParallelWalOrderedResidue;
pub use parallel_wal::ParallelWalShadowVerdict;
pub use parallel_wal::ParallelWalTraceRecord;
pub use parallel_wal::SegmentHeader;
pub use parallel_wal::SegmentRecoveryOptions;
pub use parallel_wal::SegmentRecoveryResult;
pub use parallel_wal::cleanup_segments;
pub use parallel_wal::default_parallel_wal_lane_count;
pub use parallel_wal::delete_segment;
pub use parallel_wal::list_segments;
pub use parallel_wal::max_durable_epoch;
pub use parallel_wal::parallel_wal_coordinator_for_path;
pub use parallel_wal::parallel_wal_fallback_reason_name;
pub use parallel_wal::parallel_wal_mode_name;
pub use parallel_wal::parallel_wal_shadow_verdict_name;
pub use parallel_wal::parallel_wal_should_shadow_compare;
pub use parallel_wal::read_segment;
pub use parallel_wal::recover_and_apply_segments;
pub use parallel_wal::recover_segments;
pub use parallel_wal::remove_parallel_wal_coordinator;
pub use parallel_wal::resolve_parallel_wal_control_surface_from_env;
pub use parallel_wal::segment_path;
pub use parallel_wal::write_segment;
pub use per_core_buffer::AppendOutcome;
pub use per_core_buffer::BufferConfig;
pub use per_core_buffer::BufferState;
pub use per_core_buffer::DEFAULT_BUFFER_SLOT_COUNT;
pub use per_core_buffer::EpochConfig;
pub use per_core_buffer::EpochFlushBatch;
pub use per_core_buffer::EpochOrderCoordinator;
pub use per_core_buffer::FallbackDecision;
pub use per_core_buffer::OverflowPolicy;
pub use per_core_buffer::PerCoreWalBuffer;
pub use per_core_buffer::PerCoreWalBufferPool;
pub use per_core_buffer::WalRecord;
pub use per_core_buffer::thread_buffer_slot;
pub use recovery_fence::CheckpointChecksumVerdict;
pub use recovery_fence::ExpectedPageChecksum;
pub use recovery_fence::PidOwnedLockEntry;
pub use recovery_fence::PidOwnedLockRegistry;
pub use recovery_fence::RECOVERY_FENCE_BACKOFF;
pub use recovery_fence::RECOVERY_FENCE_MAX_RETRIES;
pub use recovery_fence::RecoveryFence;
pub use recovery_fence::RecoveryFenceGuard;
pub use recovery_fence::ensure_db_fsync_before_wal_truncate;
pub use recovery_fence::execute_recovery_barrier;
pub use recovery_fence::fsync_db_file_full;
pub use recovery_fence::pid_alive_os;
pub use recovery_fence::verify_checkpoint_checksum_prefix;
pub use telemetry::NoOpWalObserver;
pub use telemetry::WalTelemetryEvent;
pub use telemetry::WalTelemetryObserver;
pub use telemetry::WalTelemetryRingBuffer;
pub use telemetry::WalTelemetrySnapshot;
pub use telemetry::wal_telemetry_snapshot;
pub use wal::WalFile;
pub use wal::WalGenerationIdentity;
pub use wal_fec::DEFAULT_RAPTORQ_REPAIR_SYMBOLS;
pub use wal_fec::MAX_RAPTORQ_REPAIR_SYMBOLS;
pub use wal_fec::WAL_FEC_GROUP_META_MAGIC;
pub use wal_fec::WAL_FEC_GROUP_META_VERSION;
pub use wal_fec::WalFecDecodeProof;
pub use wal_fec::WalFecGroupId;
pub use wal_fec::WalFecGroupMeta;
pub use wal_fec::WalFecGroupMetaInit;
pub use wal_fec::WalFecGroupRecord;
pub use wal_fec::WalFecRecoveredGroup;
pub use wal_fec::WalFecRecoveryConfig;
pub use wal_fec::WalFecRecoveryFallbackReason;
pub use wal_fec::WalFecRecoveryLog;
pub use wal_fec::WalFecRecoveryOutcome;
pub use wal_fec::WalFecRepairEvent;
pub use wal_fec::WalFecRepairEvidenceCard;
pub use wal_fec::WalFecRepairEvidenceQuery;
pub use wal_fec::WalFecRepairMetricsSnapshot;
pub use wal_fec::WalFecRepairPipeline;
pub use wal_fec::WalFecRepairPipelineConfig;
pub use wal_fec::WalFecRepairPipelineStats;
pub use wal_fec::WalFecRepairSeverityBucket;
pub use wal_fec::WalFecRepairSeverityHistogram;
pub use wal_fec::WalFecRepairSource;
pub use wal_fec::WalFecRepairWitnessTriple;
pub use wal_fec::WalFecRepairWorkItem;
pub use wal_fec::WalFecScanResult;
pub use wal_fec::WalFrameCandidate;
pub use wal_fec::append_wal_fec_group;
pub use wal_fec::build_source_page_hashes;
pub use wal_fec::ensure_wal_with_fec_sidecar;
pub use wal_fec::find_wal_fec_group;
pub use wal_fec::generate_wal_fec_repair_symbols;
pub use wal_fec::identify_damaged_commit_group;
pub use wal_fec::persist_wal_fec_raptorq_repair_symbols;
pub use wal_fec::query_raptorq_repair_evidence;
pub use wal_fec::raptorq_repair_events_snapshot;
pub use wal_fec::raptorq_repair_evidence_snapshot;
pub use wal_fec::raptorq_repair_metrics_snapshot;
pub use wal_fec::read_wal_fec_raptorq_repair_symbols;
pub use wal_fec::record_raptorq_recovery_log;
pub use wal_fec::recover_wal_fec_group_with_config;
pub use wal_fec::recover_wal_fec_group_with_decoder;
pub use wal_fec::recovery_log_from_outcome;
pub use wal_fec::reset_raptorq_repair_telemetry;
pub use wal_fec::scan_wal_fec;
pub use wal_fec::wal_fec_path_for_wal;
pub use wal_index::WAL_CKPT_INFO_BYTES;
pub use wal_index::WAL_CKPT_LOCK;
pub use wal_index::WAL_INDEX_HASH_MASK;
pub use wal_index::WAL_INDEX_HASH_MULTIPLIER;
pub use wal_index::WAL_INDEX_HASH_SLOTS;
pub use wal_index::WAL_INDEX_HDR_BYTES;
pub use wal_index::WAL_INDEX_PAGE_ARRAY_ENTRIES;
pub use wal_index::WAL_INDEX_VERSION;
pub use wal_index::WAL_LOCK_SLOT_COUNT;
pub use wal_index::WAL_READ_LOCK_BASE;
pub use wal_index::WAL_READ_MARK_COUNT;
pub use wal_index::WAL_RECOVER_LOCK;
pub use wal_index::WAL_SHM_FIRST_HEADER_BYTES;
pub use wal_index::WAL_SHM_FIRST_HEADER_U32_SLOTS;
pub use wal_index::WAL_SHM_FIRST_USABLE_PAGE_ENTRIES;
pub use wal_index::WAL_SHM_HASH_BYTES;
pub use wal_index::WAL_SHM_PAGE_ARRAY_BYTES;
pub use wal_index::WAL_SHM_SEGMENT_BYTES;
pub use wal_index::WAL_SHM_SUBSEQUENT_USABLE_PAGE_ENTRIES;
pub use wal_index::WAL_WRITE_LOCK;
pub use wal_index::WalCkptInfo;
pub use wal_index::WalHashLookup;
pub use wal_index::WalIndexHashSegment;
pub use wal_index::WalIndexHdr;
pub use wal_index::WalIndexSegmentKind;
pub use wal_index::decode_native_u32;
pub use wal_index::encode_native_u32;
pub use wal_index::parse_shm_header;
pub use wal_index::simple_modulo_slot;
pub use wal_index::usable_page_entries;
pub use wal_index::wal_index_hash_slot;
pub use wal_index::wal_index_hdr_copies_match;
pub use wal_index::write_shm_header;

Modules§

cell_delta_commit
Cell-Delta WAL Commit Integration (C4: bd-l9k8e.4)
cell_delta_wal
Cell-Delta WAL Frame Format (C4-WAL: bd-l9k8e.10)
checkpoint
WAL checkpoint planning primitives for PASSIVE/FULL/RESTART/TRUNCATE modes.
checkpoint_executor
WAL checkpoint execution engine.
checksum
WAL checksum and integrity helpers.
group_commit
Group commit with consolidation for WAL frame writes (bd-ncivz.3).
metrics
WAL observability metrics.
native_commit
Native mode commit protocol (§7.11).
parallel_wal
Parallel WAL coordinator (D1: bd-3wop3.1).
per_core_buffer
recovery_compaction
Native mode recovery algorithm (§7.12) and ECS storage reclamation (§7.13).
recovery_fence
Recovery fencing, dead-PID lock release, and checkpoint checksum validation primitives for bd-yfdb6 (OPS-3-2 CRITICAL).
telemetry
Machine-validated WAL telemetry schema.
wal
Core WAL file I/O layer.
wal_fec
WAL-FEC sidecar format (.wal-fec) for self-healing WAL durability (§3.4.1).
wal_index
WAL-index hash table primitives.