HiveBrain v1.2.0
Get Started
← Back to all entries
snippetbashTip

bq — A Python-based tool for BigQuery, Google Cloud's fully managed and completely serverless enterprise

Submitted by: @import:tldr-pages··
0
Viewed 0 times
bqcommandpythonclitoolbigquerybasedfor

Problem

How to use the bq command: A Python-based tool for BigQuery, Google Cloud's fully managed and completely serverless enterprise data warehouse. More information: <https://docs.cloud.google.com/bigquery/docs/reference/bq-cli-reference>.

Solution

bq — A Python-based tool for BigQuery, Google Cloud's fully managed and completely serverless enterprise data warehouse. More information: <https://docs.cloud.google.com/bigquery/docs/reference/bq-cli-reference>.

Run query against a BigQuery table using standard SQL, add --dry_run flag to estimate the number of bytes read by the query:
bq query --nouse_legacy_sql 'SELECT COUNT(*) FROM {{DATASET_NAME}}.{{TABLE_NAME}}'


Run a parameterized query:
bq query --use_legacy_sql=false --parameter='ts_value:TIMESTAMP:2016-12-07 08:00:00' 'SELECT TIMESTAMP_ADD(@ts_value, INTERVAL 1 HOUR)'


Create a new dataset or table in the US location:
bq mk --location=US {{dataset_name}}.{{table_name}}


List all datasets in a project:
bq ls --filter labels.{{key}}:{{value}} --max_results {{integer}} --format=prettyjson --project_id {{project_id}}


Batch load data from a specific file in formats such as CSV, JSON, Parquet, and Avro to a table:
bq load --location {{location}} --source_format {{CSV|JSON|PARQUET|AVRO}} {{dataset}}.{{table}} {{path/to/source}}


Copy one table to another:
bq cp {{dataset}}.{{OLD_TABLE}} {{dataset}}.{{new_table}}


Display help:
bq help

Code Snippets

Run query against a BigQuery table using standard SQL, add `--dry_run` flag to estimate the number of bytes read by the query

bq query --nouse_legacy_sql 'SELECT COUNT(*) FROM {{DATASET_NAME}}.{{TABLE_NAME}}'

Run a parameterized query

bq query --use_legacy_sql=false --parameter='ts_value:TIMESTAMP:2016-12-07 08:00:00' 'SELECT TIMESTAMP_ADD(@ts_value, INTERVAL 1 HOUR)'

Create a new dataset or table in the US location

bq mk --location=US {{dataset_name}}.{{table_name}}

List all datasets in a project

bq ls --filter labels.{{key}}:{{value}} --max_results {{integer}} --format=prettyjson --project_id {{project_id}}

Batch load data from a specific file in formats such as CSV, JSON, Parquet, and Avro to a table

bq load --location {{location}} --source_format {{CSV|JSON|PARQUET|AVRO}} {{dataset}}.{{table}} {{path/to/source}}

Context

tldr-pages: common/bq

Revisions (0)

No revisions yet.