Struct TransactionalBinaryWriter 

pub struct TransactionalBinaryWriter { /* private fields */ }

Transactional binary writer providing ACID guarantees for concurrent chunk operations.

The TransactionalBinaryWriter manages the complex process of writing multiple data chunks to a file while maintaining transactional integrity. It supports high-concurrency scenarios where multiple chunks can be written simultaneously from different threads or async tasks.

ACID Guarantees

Atomicity

Either all chunks are successfully written and committed, or no changes are made to the final output file. Partial writes are isolated in temporary files until the complete transaction is ready for commit.

Consistency

The file system remains in a consistent state throughout the operation. Temporary files are used to prevent corruption of the final output.

Isolation

Concurrent chunk writes do not interfere with each other. Each chunk is written to its designated position without affecting other chunks.

Durability

Once committed, the written data survives system crashes and power failures. Data is properly flushed to disk before the transaction is considered complete.

Core Capabilities

• Transactional Semantics: All-or-nothing commit behavior
• Concurrent Writing: Multiple chunks written simultaneously
• Progress Tracking: Real-time monitoring of write completion
• Crash Recovery: Checkpoint-based recovery mechanisms
• Resource Management: Automatic cleanup of temporary resources

Thread Safety

All operations are thread-safe and can be called concurrently:

• File access is protected by Arc<Mutex<File>>
• Progress counters use atomic operations
• Chunk tracking uses mutex-protected HashSet
• No data races or undefined behavior in concurrent scenarios

Implementations

impl TransactionalBinaryWriter

pub async fn new( output_path: PathBuf, expected_chunk_count: u64, ) -> Result<Self, PipelineError>

Creates a new transactional binary writer.

Arguments

• output_path - Final path where the file will be written
• expected_chunk_count - Total number of chunks expected

Returns

• Result<Self, PipelineError> - New writer or error

pub async fn commit(self) -> Result<(), PipelineError>

Commits all written chunks atomically.

This method validates that all expected chunks have been written, flushes data to disk, and atomically moves the temporary file to the final output location.

Returns

• Result<(), PipelineError> - Success or error

pub async fn rollback(self) -> Result<(), PipelineError>

Rolls back the transaction and cleans up temporary files.

Returns

• Result<(), PipelineError> - Success or error

pub async fn progress(&self) -> (u64, u64, u64)

Returns the current progress of the transaction.

Returns

• (completed_chunks, total_expected, bytes_written) - Progress information

pub async fn is_complete(&self) -> bool

Checks if the transaction is complete (all chunks written).

pub fn total_chunks(&self) -> u64

Returns the total number of chunks expected.

pub fn progress_percentage(&self) -> f64

Returns the progress as a percentage.

pub fn is_transaction_active(&self) -> bool

Checks if a transaction is currently active.

Trait Implementations

impl BinaryFormatWriter for TransactionalBinaryWriter

Implement BinaryFormatWriter trait for TransactionalBinaryWriter

fn write_chunk(&mut self, chunk: ChunkFormat) -> Result<(), PipelineError>

Writes a processed chunk (compressed/encrypted data) to the .adapipe file

fn write_chunk_at_position<'life0, 'async_trait>( &'life0 self, chunk: ChunkFormat, sequence_number: u64, ) -> Pin<Box<dyn Future<Output = Result<(), PipelineError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Writes a processed chunk at a specific position for concurrent processing

fn finalize<'life0, 'async_trait>( &'life0 self, final_header: FileHeader, ) -> Pin<Box<dyn Future<Output = Result<u64, PipelineError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Finalizes the .adapipe file by writing the footer with complete metadata

fn bytes_written(&self) -> u64

Gets the current number of bytes written

fn chunks_written(&self) -> u32

Gets the current number of chunks written

impl Drop for TransactionalBinaryWriter

Implement Drop to ensure cleanup on panic or early termination

fn drop(&mut self)

Executes the destructor for this type.