using test is a must but looking into the debug log of the jobs where I added some tests to some models in the related yml I see something I would like to have confirmation on:
create table
"dwh"."data"."traffic__dbt_tmp"
...
alter table "dwh"."data"."traffic__dbt_tmp" rename to "traffic"
...
test.dwh.not_null_traffic_user_id
why does not do test on the tmp and exit in case of failure but it does on the final model?
@obar1 what command did you use to invoke dbt?
If you used dbt run, then it will create the models without testing them. If you used dbt build, it will create the models and then test them, and if they fail they will prevent downstream models from being built.
Running tests against the temp model and only committing if they pass is an interesting idea; I suspect the answer to your question is that the majority of dbt’s models are views instead of tables so don’t go through this interstitial state.
As noted above, dbt build will stop downstream models from being built if the test fails, but that still means that the failing model is in prod.
From what you’re describing, you might want to look at Performing a blue/green deploy of your dbt project on Snowflake; a blue/green deploy is where you “swap out” the production version of your tables only after all the tests have passed.
Coming back to this - check out Provide option to allow tests to be ran before a model's data is updated · Discussion #5687 · dbt-labs/dbt-core · GitHub for the discussion around whether this is added to dbt Core in the future!
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.