generate_file_path

Function generate_file_path 

Source
pub fn generate_file_path(
    data_location: &str,
    partition_path: Option<String>,
) -> String
Expand description

Generates a unique file path for a Parquet data file.

This function creates a unique file path by combining the data location, partition path, and a UUID-based filename. If no partition path is provided, it generates a random directory path using hex-encoded random bytes.

§Arguments

  • data_location - Base directory where data files should be stored
  • partition_path - Optional partition path component (e.g., “year=2024/month=01/”)

§Returns

  • String - Complete file path ending with “.parquet”

§File Path Structure

The generated path follows this pattern:

  • With partition: {data_location}/{partition_path}{uuid}.parquet
  • Without partition: {data_location}/{random_hex}/{uuid}.parquet

§Examples

use iceberg_rust::arrow::write::generate_file_path;

// With partition path
let path1 = generate_file_path("/data", Some("year=2024/month=01/".to_string()));
// Result: "/data/year=2024/month=01/01234567-89ab-cdef-0123-456789abcdef.parquet"

// Without partition path (generates random directory)
let path2 = generate_file_path("/data", None);
// Result: "/data/a1b/01234567-89ab-cdef-0123-456789abcdef.parquet"

§Implementation Details

  • Uses cryptographically secure random bytes for UUID generation
  • Creates a UUID v1 timestamp-based identifier for uniqueness
  • Random directory names use 3 bytes of entropy (6 hex characters)
  • Automatically strips path prefixes using strip_prefix()