Hi there!
I was searching for a way to prevent the user to run the whole project accidentally. For example, for very large projects, if someone accidentally does a dbt run
dbt can try to run thousands of models and it can be very costly.
Of course, you can interrupt it with ctrl-c, but I wanted something safer (the user can just run the command and go drink a coffee), so I created this very simple macro, that can be used with the on-run-start hook.
macros/check_select_arg.sql
{% macro check_select_arg() %}
{% if not invocation_args_dict.select and target.name != 'prod' %}
{{ exceptions.raise_compiler_error("Error: You must provide at least one select argument") }}
{% endif %}
{% endmacro %}
dbt_project.yml
on-run-start:
- "{{ check_select_arg() }}"
It checks if the user has invoked some command with the select
argument, and if they don’t, an error will be raised.
You can allow some specific targets to run commands without this restriction, to configure production jobs easier, for example.
You can customize this macro the way it makes more sense to you.
3 Likes
Hi Bruno,
for some unexpected reason, this macro is executed only once.
I explain:
dbt run => BOOM => exception
if i do dbt run again, this time I don’t enter at all inside the macro (I’ve tried to output some log instruction at the highest available level.
Is is intended ? How can we do to ensure this check is executed everytime we execute a run (dbt 1.4.5)
Hi @mickaelandrieu, not sure I got it.
I try running it twice here and it worked
Can you give me more details? Did you added the macro to on-run-start?
Yes,
this way:
# dbt_project.yml
on-run-start: "{{ check_select_args() }}"
Here is “my” french version:
{% macro check_select_args() %}
{% do log('ON START ✅', True) %}
{% if not invocation_args_dict.select and target.name != 'prod' %}
{{ exceptions.raise_compiler_error("❌ Tu dois spécifier le ou les fichiers à reconstruire à l'aide de l'argument -s : dbt run -s tags:jobready ") }}
{% endif %}
{% endmacro %}
I don’t know why it’s not triggered at all, even the log message:
❯ dbt run
21:00:50 Running with dbt=1.4.5
21:00:50 [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 2 unused configuration paths:
- models.yyy
- models.xxx
21:00:50 Found 161 models, 132 tests, 3 snapshots, 0 analyses, 560 macros, 0 operations,
18 seed files, 73 sources, 9 exposures, 0 metrics
21:00:51
21:00:52 Concurrency: 4 threads (target='dev')
21:00:52
21:00:52 1 of 161 START sql table model [Cant add more info, enterprise project]
And I’ve renamed the macro to add an “s” both on the file name (useless) and on the macro name (required).
I’ll sleep thinking on that and maybe tomorrow I’ll have the answer 
1 Like
That’s weird, I’ve tested your version and it worked.
Is your macro is inside the macros/ folder?
Yes !
Funny thing: just upgraded to the latest 1.5 (1.5.4) and … it works. Meh…
I don’t like when I don’t understand.
Though, the latest 1.6 produce an error, so I can’t upgrade atm => “cannot import name ‘POLLING_PREDICATE’ from ‘google.api_core.future.polling’”
But this is another issue for another post ! Thanks for your help Bruno, and honored to meet you (even if it’s virtual) : I’m an happy reader of your tips on LinkedIn 
1 Like
This was very helpful. We also noticed that dbt compile
calls dbt run
. Usually compile is a cheap operation that can be used to test massive refactors. This would prevent project wide compile.
If you want this check to bypass compile and only apply for dbt run
and dbt build
you can do this,
{% macro check_select_args() %}
{% if not invocation_args_dict.select and (target.name != 'prod') and (invocation_args_dict.which in ['run', 'build'])%}
{{ exceptions.raise_compiler_error("run/build should have select argument. ") }}
{% endif %}
{% endmacro %}
2 Likes
Awesome addition to the macro @adiamaan92 !! Loved that
This way the admin can choose which commands will follow this rule. I would add test, source and snapshot, and we can also make a list for the target names
{% macro check_select_arg() %}
{% if
not invocation_args_dict.select
and target.name not in ['prod']
and invocation_args_dict.which in ['build', 'run', 'test', 'source', 'snapshot']
%}
{{ exceptions.raise_compiler_error("Error: You must provide at least one select argument") }}
{% endif %}
{% endmacro %}
2 Likes