Dear Community.
The problem I’m having is that every CI Job builds all models when we are only changing one SQL file.
Our job command is <dbt build --select state:modified+>
I was under the impression that this would only check the modified model and its downstream relations, but for every PR we push to github, the check goes over our 240+ models, which means a large part of getting the PR to production is waiting for the check to be finished.
The context of why I’m trying to do this
We’re trying to limit the time we have to wait for the CI check to bring small changes to production.
What I’ve already tried
I’ve found this helpfull video: dbt state comparison example + dbt Cloud Slim CI | Loom
and according to the video a model will be considered modified when some setting in the node differs from production. Since we build our models in staging, and check it with production, would that result in all our models always having to be checked?
Furthermore the Database and schema are always different from production in our PRs, because our ci jobs run on a different database. That is how a previous colleague has set things up.
The same with the materialization: In our CI job we materialize as view , where in production the materialization is table.
These are some of our settings that might be limiting our CI jobs to only run the modified file, but I’m not sure what is the cause is in our set up.
Is there anyone here who completely understands how this works and can give me some advice on the changes we should make to be able to fully profit of SLIM CI?
Thanks in advance,
Kind regards,
Luca