faucet-common-bigquery 1.0.2

Shared credentials and client construction for the faucet-stream BigQuery source and sink connectors
Documentation

faucet-common-bigquery

Crates.io Docs.rs MSRV License

Shared Google BigQuery credentials and client construction for the BigQuery source and sink connectors. Part of the faucet-stream ecosystem.

This is a small internal/shared crate: it holds the one credential enum and the one async client builder that both faucet-source-bigquery and faucet-sink-bigquery need, so the two crates accept exactly the same auth shape and build their gcp_bigquery_client::Client the same way.

What it provides

Item Kind Description
BigQueryCredentials enum How to authenticate with BigQuery. Derives Serialize, Deserialize, JsonSchema so it round-trips through YAML/JSON configs and CLI schema introspection. Its Debug impl masks inline service-account JSON as *** while leaving the key path visible.
build_client(&BigQueryCredentials) async fn Turns a BigQueryCredentials into a ready-to-use [gcp_bigquery_client::Client]. Returns FaucetError::Auth on authentication failure or unparseable inline JSON.

BigQueryCredentials variants

Credentials serialize as { type: <method>, config: { … } } (adjacent tagging, snake_case discriminators) — the consistent auth wire shape shared by every faucet connector.

type config fields Description
service_account_key_path path: String Path to a service-account JSON key file on disk.
service_account_key json: String Inline service-account JSON key content (e.g. injected from a secrets manager).
application_default (none) Use Application Default Credentials — workload identity, gcloud auth application-default login, or the GOOGLE_APPLICATION_CREDENTIALS env var.

Example YAML, as it appears inside a source/sink credentials: block:

# Key file on disk
credentials:
  type: service_account_key_path
  config:
    path: /etc/faucet/bq-key.json

# Inline JSON (often via a secret reference)
credentials:
  type: service_account_key
  config:
    json: ${secret:BQ_SA_JSON}

# Application Default Credentials — no config
credentials:
  type: application_default

Who should depend on this

  • End users / pipeline authors — you almost certainly do not depend on this crate directly. Install faucet-source-bigquery or faucet-sink-bigquery (or the faucet CLI with the source-bigquery / sink-bigquery features); both re-export BigQueryCredentials, so your imports don't change.
  • Connector authors — depend on this crate if you are building a third-party faucet-source-* / faucet-sink-* crate that talks to BigQuery and want to accept the same credentials shape and reuse the same client builder.

Installation

cargo add faucet-common-bigquery

The only required faucet dependency is faucet-core (pulled in transitively); see the Third-Party Connector Friendliness guidance.

Usage

use faucet_common_bigquery::{build_client, BigQueryCredentials};

#[tokio::main]
async fn main() -> Result<(), faucet_core::FaucetError> {
    // Built from your connector's deserialized config.
    let creds = BigQueryCredentials::ServiceAccountKeyPath {
        path: "/etc/faucet/bq-key.json".to_string(),
    };

    let client = build_client(&creds).await?;
    // `client` is a `gcp_bigquery_client::Client` — drive jobs / streaming inserts with it.
    let _ = client;
    Ok(())
}

Embed BigQueryCredentials in your own connector config so it deserializes from the standard wire shape:

use faucet_common_bigquery::BigQueryCredentials;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, JsonSchema)]
pub struct MyBigQueryConfig {
    pub project_id: String,
    pub credentials: BigQueryCredentials,
}

Feature flags

None. The crate has no optional features; all functionality is always compiled.

Troubleshooting

Symptom Cause / fix
FaucetError::Auth("invalid service account JSON: …") The service_account_key json string is not valid service-account JSON. Verify the value (and that any secret reference resolved to the full key body, not a path).
FaucetError::Auth("BigQuery auth failed: …") The key file path is wrong/unreadable, the service account lacks BigQuery permissions, or ADC could not be resolved. Check the path, the SA's IAM roles, and run gcloud auth application-default login for the application_default variant.
Inline key shows as ServiceAccountKey(***) in logs Intended — the Debug impl masks inline JSON so it never leaks into tracing output. Key paths are not masked.
application_default can't find credentials Set GOOGLE_APPLICATION_CREDENTIALS, run gcloud auth application-default login, or run on a GCP runtime with an attached service account / workload identity.

See also

License

Licensed under either of

at your option.