The problem I’m having .
I need to execute numerous release checks on different dbt projects (over 20). To achieve this I define a macro with the release check logic and in a DevOps pipeline I call this macro with the dbt run-operation
command. It works perfectly when the specific dbt project does not use any variables but when it does I get error.
The context of why I’m trying to do this
In my task, I need to only execute dbt run-operation
with the logic I defined in the macro. I don’t need to bother about what variables the different dbt projects are using as this is related to their project and I simply do some checks on the model.
What I’ve already tried
I know some of the solutions are:
- pass the variable in my
dbt run-operation
command - The variables are custom and can be as many as needed. The process I am building needs to be automated and not dependent on custom variables - set default variable - the dbt projects are over 20 and it is not possible to force all developers to set default values when even my code doesn’t need them and because during the parsing phase, dbt tries to resolve all variables defined in the project, which is causing the error.
Some example code or error messages
dbt run-operation fetch_var_aliases --args "{'model_name':'Some_model'}"
10:38:12 Running with dbt=1.7.9
10:38:12 Encountered an error:
Compilation Error
Could not render {"Function": "TnP","module": "Energy","layer": "propagation","plantid":"{{ var("dmo_plantid")}}"}: Required var 'dmo_plantid' not found in config:
Vars supplied to <Configuration> = {}