Expand description
§BigQuery for Rust
A client for Google BigQuery that uses gRPC throughout: the Storage Read API for table scans and large results, the Storage Write API for inserts, and the v2 API for queries, jobs, datasets and tables.
Full documentation, with a chapter per topic: https://bigquery-rust.abdolence.dev
§Example
use bigquery::*;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
struct Order {
id: i64,
customer: String,
total: f64,
}
const SHOP: BigQueryDatasetId = BigQueryDatasetId::from_static("shop");
const ORDERS: BigQueryTableId = BigQueryTableId::from_static("orders");
let db = BigQueryDb::new("my-project-id").await?;
// Insert through the Storage Write API
let orders = vec![Order {
id: 1,
customer: "Alice".to_string(),
total: 120.0,
}];
db.fluent()
.insert()
.into(SHOP.table(ORDERS))
.objects(&orders)
.execute()
.await?;
// Read the table back through the Storage Read API, with a typed filter
let large_orders: Vec<Order> = db
.fluent()
.select()
.from(SHOP.table(ORDERS))
.filter(|filter| filter.field(path!(Order::total)).gt(100.0))
.obj()
.query()
.await?;
// Query with parameters
let alice_orders: Vec<Order> = db
.fluent()
.query("SELECT id, customer, total FROM shop.orders WHERE customer = @customer")
.param("customer", "Alice")
.obj()
.query()
.await?;Re-exports§
pub extern crate struct_path;
Re-exports§
pub use gcloud_sdk;pub use arrow_array;pub use arrow_schema;pub use jiff;pub use url;
Modules§
- errors
- The error types of this crate, built around
BigQueryError. The crate’s error type and the pieces it is built from. - serialize_
as_ date #[serde(with = "bigquery::serialize_as_date")]for ajiff::civil::Datefield, the same asBigQueryDate.- serialize_
as_ datetime #[serde(with = "bigquery::serialize_as_datetime")]for ajiff::civil::DateTimefield, the same asBigQueryDateTime.- serialize_
as_ decimal #[serde(with = "bigquery::serialize_as_decimal")]for a bare decimal field, the same asBigQueryDecimal.- serialize_
as_ json #[serde(with = "bigquery::serialize_as_json")]for a bare field of a JSON column, the same asBigQueryJson.- serialize_
as_ optional_ date #[serde(with = "bigquery::serialize_as_optional_date")]for anOption<jiff::civil::Date>field, the same asOption<BigQueryDate>.- serialize_
as_ optional_ datetime #[serde(with = "bigquery::serialize_as_optional_datetime")]for anOption<jiff::civil::DateTime>field, the same asOption<BigQueryDateTime>.- serialize_
as_ optional_ decimal #[serde(with = "bigquery::serialize_as_optional_decimal")]for anOptiondecimal field, the same asOption<BigQueryDecimal<T>>.- serialize_
as_ optional_ json #[serde(with = "bigquery::serialize_as_optional_json")]for anOptionfield of a JSON column, the same asOption<BigQueryJson<T>>.- serialize_
as_ optional_ time #[serde(with = "bigquery::serialize_as_optional_time")]for anOption<jiff::civil::Time>field, the same asOption<BigQueryTime>.- serialize_
as_ optional_ timestamp #[serde(with = "bigquery::serialize_as_optional_timestamp")]for anOption<jiff::Timestamp>field, the same asOption<BigQueryTimestamp>.- serialize_
as_ time #[serde(with = "bigquery::serialize_as_time")]for ajiff::civil::Timefield, the same asBigQueryTime.- serialize_
as_ timestamp #[serde(with = "bigquery::serialize_as_timestamp")]for ajiff::Timestampfield, the same asBigQueryTimestamp.- testing
- A fake BigQuery for your tests.
Macros§
- path
- Builds the BigQuery column path for one field of
Structas aString. - path_
camel_ case - Builds the BigQuery column path for one field of
Struct, converting it to camelCase. - paths
- Builds the BigQuery column paths for several fields of
Structas aVec<String>. - paths_
camel_ case - Builds the BigQuery column paths for several fields of
Struct, converting each to camelCase. - sql_
file - Embeds a GoogleSQL file as a
BigQuerySqland checks, as the calling crate compiles, that the names listed after the path are exactly the file’s@nameparameters.
Structs§
- BigQuery
Batch Rows - The rows of one Arrow
RecordBatchdecoded intoT, one item per row, in order. - BigQuery
CdcWriter - A writer of CDC changes: each row carries
_CHANGE_TYPEand, optionally,_CHANGE_SEQUENCE_NUMBER, and BigQuery applies it to the row with the same primary key. - BigQuery
Change - One CDC change of a row.
- BigQuery
Change Sequence Number - A CDC
_CHANGE_SEQUENCE_NUMBER, which orders changes to one key.From<u64>writes it in hex; any other text is sent as it is, and BigQuery checks its form. - BigQuery
Dataset - A dataset as
GetDatasetreturns it. - BigQuery
Dataset Builder - One dataset, from
BigQuerySchemaBuilder::dataset. End it with one of its calls. - BigQuery
Dataset Create Builder - A new dataset’s settings, from
BigQueryDatasetBuilder::create. - BigQuery
Dataset Id - A BigQuery dataset ID, checked only for what would let it reshape a path or a query.
- BigQuery
Dataset List Builder - A listing of the datasets in a project, from
BigQuerySchemaBuilder::datasets. - BigQuery
Dataset Ref - A dataset, by project and dataset ID.
- BigQuery
Dataset Summary - A dataset as
ListDatasetsreturns it, which is less thanBigQueryDataset: read one withBigQueryDatasetBuilder::getfor the rest. - BigQuery
Dataset Update Builder - Changes to a dataset, from
BigQueryDatasetBuilder::update. What it does not change stays as the dataset has it. - BigQuery
Date - A DATE column with the codecs’ integer fast path; any other format sees it as a plain
jiff::civil::Date, ori32days since 1970-01-01 in formats that are not human-readable. - BigQuery
Date Time - A DATETIME column with the codecs’ integer fast path; any other format sees it as a plain
jiff::civil::DateTime, ori64civil microseconds since 1970-01-01T00:00:00 in formats that are not human-readable. - BigQuery
Db - The main entry point for working with Google BigQuery.
- BigQuery
DbOptions - Configuration options for the
BigQueryDbclient. - BigQuery
DbOptions Init - Required fields of
BigQueryDbOptions; convert withBigQueryDbOptions::fromor.into(). - BigQuery
Decimal - A NUMERIC or BIGNUMERIC value held in any decimal type that round-trips through its
DisplayandFromStrtext, such asbigdecimal::BigDecimalorrust_decimal::Decimal. - BigQuery
Decimal Params - The declared precision and scale of a NUMERIC or BIGNUMERIC column. A declared precision
without a scale has scale 0, so
NUMERIC(10)equalsNUMERIC(10, 0). - BigQuery
Delete Initial Builder - The first stage of a delete, from
BigQueryExprBuilder::delete. - BigQuery
Delete Keys Builder - A delete of rows by primary key values, written as CDC deletes of rows that hold only the key columns.
- BigQuery
Delete ObjBuilder - A delete of rows by primary key, written as CDC deletes through the table’s default stream.
- BigQuery
Delete Table Builder - A delete with its table; continue with the keys of the rows to delete, as plain values or as rows.
- BigQuery
DmlStats - The rows a DML statement changed, by kind.
- BigQuery
DryRun Result - What a dry run reports without running the statement.
- BigQuery
Expr Builder - The entry point for fluent BigQuery operations, obtained from
BigQueryDb::fluent(). - BigQuery
Field Schema - One column, or one field of a STRUCT.
- BigQuery
Filter - A row filter from
BigQueryFilterBuilder. It holds the first error any of its parts met, which the read’s terminal returns. - BigQuery
Filter Builder - Builds row filters: the closure given to
filterreceives one. - BigQuery
Filter Field Expr - A column targeted by
BigQueryFilterBuilder::field: pick the condition. - BigQuery
Finalized Stream - A finalized pending stream, ready for a commit together with other streams.
- BigQuery
Insert Changes Builder - An insert of CDC changes through the default stream.
- BigQuery
Insert Initial Builder - The first stage of an insert, from
BigQueryExprBuilder::insert. - BigQuery
Insert ObjBuilder - An insert of rows. The default writes through the table’s default stream, at least once.
- BigQuery
Insert Params - What an insert or a streaming writer writes to and how.
- BigQuery
Insert Params Init - Required fields of
BigQueryInsertParams; convert withBigQueryInsertParams::fromor.into(). - BigQuery
Insert Record Batches Builder - An insert of Arrow record batches. The default writes through the table’s default stream, at least once.
- BigQuery
Insert Table Builder - An insert with its table; continue with the rows or the changes to write.
- BigQuery
Interval - An INTERVAL value as its three independent parts, each with its own sign, the way BigQuery
and Arrow’s
MonthDayNanohold it. - BigQuery
Job - A job as
GetJoborListJobsreturns it. - BigQuery
JobId - A BigQuery job ID.
- BigQuery
JobRef - A BigQuery job.
- BigQuery
JobStats - What a query used, from the responses the query already received; the
_with_statsterminals return it with the rows. - BigQuery
Json - A JSON value carried as its JSON text, whatever the serde format.
- BigQuery
Labels - The labels of a dataset, a table or a job: keys and values, ordered by key.
- BigQuery
Labels Into Iter - The owning iterator of
BigQueryLabels, ordered by key. - BigQuery
List Jobs Params - Which jobs
BigQueryDb::stream_jobslists, newest first. - BigQuery
Location - A BigQuery location, such as
US,EUoreurope-west1. - BigQuery
Param Type - The type of a query parameter, for a value whose type cannot be inferred.
- BigQuery
Query Builder - A GoogleSQL query being built, from
BigQueryExprBuilder::query: add parameters and job settings, then pick a terminal. - BigQuery
Query Destination - The table a query writes its result into, and what happens to the rows the table already holds.
- BigQuery
Query Id - The ID BigQuery reports for a query it answered, with or without a job.
- BigQuery
Query ObjBuilder - A typed query: pick a terminal.
- BigQuery
Query Params - What a query sends: the statement, its parameters and the job settings, as
BigQueryQueryBuildercollects them. - BigQuery
Query Params Init - Required fields of
BigQueryQueryParams; convert withBigQueryQueryParams::fromor.into(). - BigQuery
Range - A RANGE value of DATE, DATETIME or TIMESTAMP.
- BigQuery
Read Options - How a Storage Read session is opened.
- BigQuery
Read Params - What a table read asks Storage Read for.
- BigQuery
Read Params Init - Required fields of
BigQueryReadParams; convert withBigQueryReadParams::fromor.into(). - BigQuery
Recreate - A recreate
.sync()runs because a change is impossible in place and the declaration opts in to recreating. - BigQuery
Request Id - The idempotency key of a
Querycall: a retry that repeats it does not run the statement twice, within the window BigQuery keeps keys for. - BigQuery
Schema Builder - The schema namespace, from
BigQueryExprBuilder::schema. - BigQuery
Schema Column - One declared column, or one field of a declared RECORD, as
BigQuerySchemaColumnsBuilder::fieldstarts it orfrom_typeinfers it. - BigQuery
Schema Columns - The columns a
columnsclosure declares, in table order: the listfieldscollects, or the onefrom_typeinfers. - BigQuery
Schema Columns Builder - Builds the column list for
columnsandrecord. - BigQuery
Select Builder - A table read with its table named: add a filter or session settings, then pick a
terminal,
objfor typed rows orrecord_batchesfor raw Arrow. - BigQuery
Select Initial Builder - The first stage of a table read, from
BigQueryExprBuilder::select: pick the columns, or name the table. - BigQuery
Select ObjBuilder - A typed table read: pick a terminal.
- BigQuery
Sql - The GoogleSQL statement that
queryruns: text built at run time, or a.sqlfile embedded and checked bysql_file!. - BigQuery
Streaming Write Options - How a streaming writer opens its stream, batches rows and bounds what is in flight.
- BigQuery
Streaming Writer - A writer that streams rows of
Tinto one table over oneAppendRowsconnection. - BigQuery
Table - A table as
GetTablereturns it. - BigQuery
Table Declaration - One table’s declared schema and settings, with the sync options, checked.
- BigQuery
Table Id - A BigQuery table ID, checked only for what would let it reshape a path or a query.
- BigQuery
Table List Builder - A listing of one dataset’s tables, from
BigQueryDatasetBuilder::tables. - BigQuery
Table Plan - What
.sync()would do to one table, as.plan()returns it. - BigQuery
Table Ref - A table, by project, dataset and table ID.
- BigQuery
Table Schema - A table schema, normalised so that schemas from the v2 API, the Storage API and Arrow
compare with
==. - BigQuery
Table Schema Builder - One table’s declaration. End it with
planorsync, or with one of the plain callsgetanddelete, which act on the table and ignore whatever the chain declared. - BigQuery
Table Summary - A table as
ListTablesreturns it, without its schema and sizes: read one withBigQueryTableSchemaBuilder::getfor those. - BigQuery
Table Sync Report - What
.sync()did to one table. - BigQuery
Table Target - A whole table as
.sync()creates or recreates it: what is declared, plus whatever the table already has that the declaration leaves undeclared and is not pruned. - BigQuery
Time - A TIME column with the codecs’ integer fast path; any other format sees it as a plain
jiff::civil::Time, ori64microseconds since midnight in formats that are not human-readable. Sub-microsecond digits are dropped on write. - BigQuery
Timestamp - A TIMESTAMP column with the codecs’ integer fast path; any other format sees it as a plain
jiff::Timestamp: RFC 3339 text in human-readable formats, microseconds since the epoch in the others. - BigQuery
Trace Id - The
trace_ida writer sends with its stream, which BigQuery keeps for its own diagnostics; Google suggests the client’s name and version. - BigQuery
Update Initial Builder - The first stage of an update, from
BigQueryExprBuilder::update. - BigQuery
Update ObjBuilder - An update of whole rows by primary key, written as CDC upserts through the table’s default stream.
- BigQuery
Update Table Builder - An update with its table; continue with the rows to write.
- BigQuery
Withheld Change - A change
.sync()leaves out until the declaration opts in to it. - BigQuery
Write Response - The outcome of one acknowledged batch.
- BigQuery
Write Stream Name - The name of a write stream, as
CreateWriteStreamreturned it:projects/{p}/datasets/{d}/tables/{t}/streams/{id}. - BigQuery
Write Summary - What a finished writer wrote.
Enums§
- BigQuery
Change Type - What a CDC change does to the row with its primary key.
- BigQuery
Destination Write - What a query does when its destination table already holds rows. Each one applies only if the job succeeds, as one atomic update of the table.
- BigQuery
Dropped Data - Data a
.sync()deleted. - BigQuery
Field Mode - The mode of a column.
- BigQuery
Field Type - The type of a column, with the parameters that are part of it.
- BigQuery
JobCreation - Whether a query runs as a BigQuery job.
- BigQuery
JobState - Where a job is in its run, as BigQuery names it in
JobStatus.state:PENDING,RUNNINGorDONE. - BigQuery
JobType - What kind of work a job does, as BigQuery names it in
JobConfiguration.job_type:QUERY,LOAD,EXTRACT,COPYorUNKNOWN. - BigQuery
Missing Value - What BigQuery stores for a field that a row leaves out.
- BigQuery
Partition Unit - The unit of time partitioning.
- BigQuery
Partitioning - How a table is partitioned. Partition expiration is separate, see
partition_expiration, because it can change in place while the partitioning cannot. - BigQuery
Range Element Type - The element type of a RANGE column.
- BigQuery
Read Compression - The compression of the Arrow record batch buffers a read session sends.
- BigQuery
Recreate Method - How a table is recreated.
- BigQuery
Recreate Policy - What
.sync()may do when a change is impossible in place. - BigQuery
Refusal - Why
.sync()refuses a plan, writing nothing. - BigQuery
Schema Change - One difference between a declaration and the table, and how it is applied.
- BigQuery
Statement Type - The kind of statement a query ran, as BigQuery names it in
statement_type. - BigQuery
Table Type - What kind of table a table is, as BigQuery names it in
Table.type:TABLE,VIEW,MATERIALIZED_VIEW,EXTERNALorSNAPSHOT. - BigQuery
Withheld Reason - What a withheld change waits for.
- BigQuery
Write Mode - The write stream a writer uses.
Constants§
- BIGQUERY_
API_ URL - Google’s BigQuery v2 API endpoint: datasets, tables, jobs and queries.
- BIGQUERY_
STORAGE_ API_ URL - Google’s BigQuery Storage Read and Write API endpoint.
Traits§
- BigQuery
Filter Expr - What
BigQueryFilterBuilder::for_all,for_anyandnottake: a filter, or an optional one, whereNonestands for no condition.
Type Aliases§
- BigQuery
Instant - An exact instant in time, used by this library for the times BigQuery reports about a dataset, a table, a job or a write: creation and last modified times, a job’s start and end, a table’s expiration, a commit time, and a read’s snapshot time.
- BigQuery
Query Outcome - What a statement did, as
executereports it: the same figures a query’sBigQueryJobStatscarries. - BigQuery
Result - The result type of every fallible call in this crate.