I can’t get my production environment to work.
I have 2 environments for my dbt cloud project connected to my bigquery. One is my staging environment called “dbt_warehouse” and one is my production environment called “warehouse”. Both are connected to the same github branch. The only difference between the environments is that, in the dbt cloud environment page, I wrote “dbt_warehouse” in the Deployment Credentials for my staging environment and “warehouse” under my Production environment which both correspond to my bigquery datasets.
I have a bunch of jobs that succesfully run dbt models and publish them into my staging environment “dbt_warehouse” but when I copy those same models and run them as a production job to “warehouse”, nothing ever works.
Over the last 2 days, I’ve tried to seemingly everything but I’m unable to get a clear understanding of why this is happening. I tried creating a profiles.yml but that didn’t work and i tried writing things into my dbt_project.yml file but that didn’t pan out either.
Some example code or error messages
Invoke dbt Command
dbt run --select test_connection
01:10:09 Running dbt…
01:10:11 Unable to do partial parsing because saved manifest not found. Starting full parse.
/venv/dbt-a99d1be4351aba36c95044ba27f639798ff11bec/lib/python3.11/site-packages/dbt/task/sl/slg_client.py:16: FutureWarning: aliases are no longer used by BaseSettings to define which environment variables to read. Instead use the “env” field setting. See Settings Management - Pydantic
class dbtSavedQuerySettings(BaseSettings):
01:10:12 Found 21 models, 4 data tests, 10 sources, 605 macros
01:10:12 The selection criterion ‘test_connection’ does not match any enabled nodes
01:10:12
01:10:12 Nothing to do. Try checking your model configs and model specification args
OR HERE IS ANOTHER dbt RUN
Invoke dbt Command
dbt run --select dim_ads
dbt command failed01:03:59 Running dbt…
01:04:00 Unable to do partial parsing because saved manifest not found. Starting full parse.
/venv/dbt-a99d1be4351aba36c95044ba27f639798ff11bec/lib/python3.11/site-packages/dbt/task/sl/slg_client.py:16: FutureWarning: aliases are no longer used by BaseSettings to define which environment variables to read. Instead use the “env” field setting. See Settings Management - Pydantic
class dbtSavedQuerySettings(BaseSettings):
01:04:02 Found 21 models, 4 data tests, 10 sources, 605 macros
01:04:02
01:04:02
01:04:02 Concurrency: 4 threads (target=‘default’)
01:04:02
01:04:03 1 of 1 START sql table model warehouse.dim_ads … [RUN]
01:04:03 BigQuery adapter: https://console.cloud.google.com/bigquery?project=my-project-id&j=bq:US:c20ddec6-fc3f-4aa3-b80a-b019360eb06c&page=queryresults
01:04:03 1 of 1 ERROR creating sql table model warehouse.dim_ads … [ERROR in 0.45s]
01:04:03
01:04:03
I want to publish some models to “dbt_warehouse” and some models to “warehouse”, but I can’t figure out how to that. It only writes to one dataset.
Any advice would be greatly appreciated.