Data Warehouses
GCP BigQuery
Install Dependencies
pip install dcs-core[bigquery]
Define DataSource Connection in Configuration File
BigQuery datasource can be defined as below in the config file.
The type of the data source must be bigquery
.
Below is an example of the configuration file.
data_sources:
- name: bigquery_datasource
type: bigquery
config:
project: <gcp_project_id>
dataset: <gcp_dataset_name>
credentials_base64: <base64 encoded credentials json>
How to create base64 encoded credentials json?
To create the base64 encoded string you can use the command line tool base64
, or openssl base64
, or python -m base64