Module validation

Data validation and integrity checking module

Provides functionality for validating data integrity through checksums, format validation, and other verification methods:

• File integrity validation with multiple checksum algorithms (CRC32, SHA256, BLAKE3)
• Format-specific validation for scientific data formats
• Directory manifests for data validation
• Integrity metadata for tracking data provenance Data validation and integrity checking utilities

This module provides functionality for validating data integrity through checksums, hash verification, and other validation methods.

Features

• Checksums (CRC32, MD5, SHA-256, BLAKE3)
• File integrity validation
• Data format validation
• Integrity metadata for scientific data

Modules

formats

Format-specific validation utilities

schema_helpers

Helper function to create common schema types

Structs

DirectoryManifest

Manifest file for a directory

FormatValidator

Format validation rule with type and validation function

FormatValidatorRegistry

Registry of available format validators

IntegrityMetadata

File integrity metadata used for validation

ManifestEntry

Entry in a directory manifest

ManifestVerificationReport

Report from a manifest verification

SchemaDefinition

Schema definition for data validation

SchemaValidationError

Schema validation error

SchemaValidationResult

Result of schema validation

SchemaValidator

Schema validator that can validate data against schemas

ValidationReport

Report containing the results of a file validation check

Enums

ChecksumAlgorithm

Checksum algorithm types available for data validation

SchemaConstraint

Schema constraint for validation

SchemaDataType

Data type for schema validation

ValidationSource

Data source used for format validation

Functions

add_integrity_metadata

Add integrity metadata to an array of objects

calculate_checksum

Calculate a checksum for the provided data

calculate_crc32

Convenience function to calculate CRC32 checksum for a file

calculate_file_checksum

Calculate a checksum for a file

calculate_sha256

Convenience function to calculate SHA256 checksum for a file

create_checksum_file

Utility to create a checksum file for a data file

create_directory_manifest

Create a manifest file for a directory with checksums

generate_file_integrity_metadata

Generate integrity metadata for a file

generate_validation_report

Generate a validation report for a file

load_integrity_metadata

Load integrity metadata from a file

save_integrity_metadata

Save integrity metadata to a file

schema_from_json_schema

Build schemas from JSON Schema format

validate_file_exists_with_size

Check if a file exists and has the expected size

validate_file_integrity

Validate a file’s integrity using metadata

verify_array_integrity

Verify array integrity using metadata

verify_checksum

Verify a checksum against the provided data

verify_checksum_file

Verify a file against a checksum file

verify_file_checksum

Verify a file’s checksum