Skip to main content

Module credentials

Module credentials 

Source
Expand description

Credential vending for cloud storage access.

This module provides credential vending functionality that generates temporary, scoped credentials for accessing cloud storage. Similar to Apache Polaris’s credential vending, it supports:

  • AWS: STS AssumeRole with scoped IAM policies (requires credential-vendor-aws feature)
  • GCP: OAuth2 tokens with access boundaries (requires credential-vendor-gcp feature)
  • Azure: SAS tokens with user delegation keys (requires credential-vendor-azure feature)

The appropriate vendor is automatically selected based on the table location URI scheme:

  • s3:// for AWS
  • gs:// for GCP
  • az:// for Azure

§Configuration via Properties

Credential vendors are configured via properties with the credential_vendor. prefix.

§Properties format:

# Required to enable credential vending
credential_vendor.enabled = "true"

# Common properties (apply to all providers)
credential_vendor.permission = "read"          # read, write, or admin (default: read)

# AWS-specific properties (for s3:// locations)
credential_vendor.aws_role_arn = "arn:aws:iam::123456789012:role/MyRole"  # required for AWS
credential_vendor.aws_external_id = "my-external-id"
credential_vendor.aws_region = "us-west-2"
credential_vendor.aws_role_session_name = "my-session"
credential_vendor.aws_duration_millis = "3600000"  # 1 hour (default, range: 15min-12hrs)

# GCP-specific properties (for gs:// locations)
# Note: GCP token duration cannot be configured; it's determined by the STS endpoint
# To use a service account key file, set GOOGLE_APPLICATION_CREDENTIALS env var before starting
credential_vendor.gcp_service_account = "my-sa@project.iam.gserviceaccount.com"

# Azure-specific properties (for az:// locations)
credential_vendor.azure_account_name = "mystorageaccount"  # required for Azure
credential_vendor.azure_tenant_id = "my-tenant-id"
credential_vendor.azure_duration_millis = "3600000"  # 1 hour (default, up to 7 days)

§Example using ConnectBuilder:

ConnectBuilder::new("dir")
    .property("root", "s3://bucket/path")
    .property("credential_vendor.enabled", "true")
    .property("credential_vendor.aws_role_arn", "arn:aws:iam::123456789012:role/MyRole")
    .property("credential_vendor.permission", "read")
    .connect()
    .await?;

Structs§

VendedCredentials
Vended credentials with expiration information.

Enums§

VendedPermission
Permission level for vended credentials.

Constants§

API_KEY_HASH_PREFIX
Property key prefix for API key hash to permission mappings (short form). Format: api_key_hash.<sha256_hash> = "<permission>"
API_KEY_SALT
Common property key for API key salt (short form). Used to hash API keys before comparison: SHA256(api_key + “:” + salt)
CACHE_ENABLED
Common property key to enable credential caching (short form). Default: true. Set to “false” to disable caching.
DEFAULT_CREDENTIAL_DURATION_MILLIS
Default credential duration: 1 hour (3600000 milliseconds)
ENABLED
Common property key to explicitly enable credential vending (short form).
PERMISSION
Common property key for permission level (short form).
PROPERTY_PREFIX
Property key prefix for credential vendor properties. Properties with this prefix are stripped when using from_properties.

Traits§

CredentialVendor
Trait for credential vendors that generate temporary credentials.

Functions§

create_credential_vendor_for_location
Create a credential vendor for the specified table location based on its URI scheme.
detect_provider_from_uri
Detect the cloud provider from a URI scheme.
has_credential_vendor_config
Check if credential vending is enabled.
redact_credential
Redact a credential string for logging, showing first and last few characters.