Printing dbt invocation_command in the logs

[Ruba_AT]

The problem I’m having

So I’m using dbt core with snowflake data warehouse. I’m using dbt in some Kubeflow components in a pipeline… and when the components run, I want to be able to see which dbt command it has been invoked to run this component. I know I can show the input parameters for a pipeline… but my main question how can I print what invocation_command was triggered for dbt. I have read many documentation webpages and I found this invocation_command variable that has the value, but I could not find how I can get its value and then print it in the logs that dbt usually prints.

Thanks in advance

What I’ve already tried

I have tried dbt --debug run but then the log is too too long… I only want the regular logs, with the invocation_command value.

[joellabes]

@Ruba_AT the invocation_command inside of the invocation_args dict is exposed by the Jinja context, so you need to access it from somewhere that can render Jinja.

In your case I would put it in an on-run-start hook in dbt_project.yml:

#dbt_project.yml
name: "joel_sandbox"

[...]

on-run-start: 
  - "{{ print('Running command: ' ~ invocation_args_dict['invocation_command']) }}"

which will output:

(dbt-prod) joel@Joel-Labes joel-sandbox % dbt run -s first_model
22:16:16  Running with dbt=1.7.0
22:16:17  Registered adapter: snowflake=1.7.0
22:16:17  Unable to do partial parsing because a project config has changed
Running command: dbt run -s first_model

[Ruba_AT]

Thank you very much!!! Exactly what I needed! @joellabes
another small question… so this print is from Jinja? or is Jinja just anything inside {{ }} ?

[Ruba_AT]

I’m sorry, but I’m getting this error

Compilation Error
  Could not render {{ print('Running command: ' ~ invocation_args_dict['invocation_command']) }}: 'invocation_args_dict' is undefined

[joellabes]

Yes you’re right - Jinja is the templating language dbt uses, and its commands are enclosed in curly braces: {{ for templates and {% for commands. Jinja and macros | dbt Developer Hub

dbt has added a custom Jinja macro called print()

[joellabes]

Can you paste your full dbt_project.yml file? Make sure you do it as a code block (inside of backticks) to preserve the spaces as yaml is a space-sensitive language

[Ruba_AT]

I managed by some help from others to do it like this

"{{ log('invocation_command: ' ~ flags.INVOCATION_COMMAND, info=true) }}"

But my dbt_project.yml
is as follow

# Name your project! Project names should contain only lowercase characters and underscores.
name: 'my_project_dbt'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'data_project'

# These configurations specify where dbt should look for different types of files.
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"


models:
  my_project_dbt:
    +on-run-start: 
       - "{{ print('Running command: ' ~ invocation_args_dict['invocation_command']) }}"
    +materialized: table

[joellabes]

Try moving the on-run-start block to the root level (instead of indented under models) like this:

# Name your project! Project names should contain only lowercase characters and underscores.
name: 'my_project_dbt'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'data_project'

# These configurations specify where dbt should look for different types of files.
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"

on-run-start: 
  - "{{ print('Running command: ' ~ invocation_args_dict['invocation_command']) }}"

models:
  my_project_dbt:
    +materialized: table

[Ruba_AT]

it works now!!
Thanks so much

I did not know I can have on-run-start at the root level… I appreciate your kind help