The Problem
I’m unable to configure dbt Cloud to send production runs to a separate Google Cloud Project from the development one. This is crucial for a proper dev/prod separation in BigQuery.
Context
I’m building a data infrastructure from the ground up for a company without any existing setup. We plan to have individual datasets for each developer within a single development project in BigQuery, while the production project will have distinct datasets for each data layer (marts, intermediate, staging). My experience with Snowflake hasn’t fully translated to understanding BigQuery’s interaction with dbt Cloud, and I want to ensure we set a strong foundation from the beginning.
What I’ve Tried
I attempted to dynamically set the project name in dbt_project.yml
. However, production jobs continue to route data to the development project. Directly setting the target to the project production results in job failure.
Example Code
marts:
+materialized: table
+project: |
{%- if target.name == "development" -%}
development
{%- endif -%}
{%- if target.name != "development" -%}
+dataset: marts
{%- endif -%}
Any suggestions or guidance on how to approach this configuration would be greatly appreciated.