Hi, I would like to run dbt through Cloud Composer (which is the GCP’s managed service for Apache Airflow) and I am struggling to find a proper and secure way in order to authenticate dbt so that it can perform operations on Google Cloud BigQuery. So here’s my profile in profiles.yaml file: my-pr…

type: bigquery method: oauth dbt will then automatically use the credentials of the service account of the node pool. I have written a series of articles on how to run dbt at best on Google Cloud. Here is the link of part 1 if you are interested: [image] DBT at scale on Google Cloud — P…

How to run dbt on Cloud Composer and authenticate the service account

Help

joellabes Split this topic September 27, 2022, 4:07am 5

A post was split to a new topic: Equivalent of GKEStartPodOperator in Anthos Cloud Run with Workload Identity Federation

Topic		Replies	Views
Can DBT connect to BigQuery using Workload Identity? Help bigquery	1	1662	April 26, 2024
Running dbt in Composer using a KubernetesPodOperator Show and Tell airflow , orchestration-and-deployment , gcp-cloud-composer	7	17805	April 30, 2024
Running dbt in Airflow using GKEStartPodOperator Help gcp-cloud-composer	0	144	December 5, 2024
dbt cli on gcp cloudshell Help	12	2790	November 3, 2023
Dbt extremely slow on Airflow (Cloud Composer) Archive	3	3518	August 12, 2022

How to run dbt on Cloud Composer and authenticate the service account

Related topics