dbt test state:modified+ runs more than I expected

Help
, ,
1

The problem I’m having

I’m running dbt run with --select state:modified+ and some --exclude “tag:on_hold” on the master branch comparing with the state of production. As expected this returns “nothing to do”. However when I run the same command for dbt test then it starts running a lot of tests. How is this possible? If no nodes are selected for run, no nodes should be selected for test either right? Or am I misinterpreting things as I read that select for test works a bit differently.
The commands I used

dbt run --select state:modified+ --exclude "tag:on_hold tag:dbt-bigquery-monitoring"  --target dev --defer --state states/master/
Says nothing to do
dbt test --select state:modified+ --exclude "tag:on_hold tag:dbt-bigquery-monitoring"  --target dev --defer --state states/master/
runs a lot of tests

The context of why I’m trying to do this

We would like to have a subset of tests that is critical and should not have errors. Another subset could generate warnings, so we want to make sure that we only run those tests that we expect to run, not more.

What I’ve already tried

I’ve asked slack and there they set me up on the right track. I focused on one particular test that was selected, but should not have been and I compared the manifest files. I got the following result.

2c2
<   "database": "dev-database",
---
>   "database": "prod-database",
52,53c52,55
<   "unrendered_config": {},
<   "created_at": 1743063125.9621148,
---
>   "unrendered_config": {
>     "severity": "error"
>   },
>   "created_at": 1743058871.2417643,
55a58
>   "doc_blocks": [],

So whenever for example dbt parse is run in the cloud I see the unrendered_config with severity: error, when I run dbt parse locally I don’t have the severity in unrendered_config (i.e. it’s empty). This puzzles me and I would like to understand why this is.

In this video: dbt state comparison example + dbt Cloud Slim CI | Loom
you can see that the changes in unrendered_config actually don’t trigger a run, but for me it does. It’s clearly the config that triggers it as it only runs when I do --select state:modified.configs+. It does not run when I select the other state:modified properties.

So someone on slack responded

Right so that that dynamicity at play
In your prod - you have some logic that says the severity of test is error
But in your non-prod you don’t
Or vice versa
If any config is different for a node - that constitutes “state:modified”

So I have three questions:

  • does the change in database trigger the run/test with state:modified?
  • I can’t find anything related to the dynamicity of the severity of the error configured on production. Where can I start looking or find it?
  • What does the unrendered_config mean?