dbt core - snowflake - Running pre hook to capture query table before running dbt test

Help
, , ,
1

Running Snowflake models and tests in dbt-core

Have a pre-hook macro generate_query_tags to add query_tags in Snowflake containing DAG ID and Run ID.

dbt_projects.yml contents:

name: “financials”

version: “1.0.0”

config-version: 2

profile: “fin”
models:
financials:
pre-hook:
- “{{ generate_query_tags() }}”
+materialized: table
bronze:
+schema: “bronze”
upf:
+tags: upf
silver:
+tags: silver
+schema: “silver”
upf:
+tags: upf
gold:
+tags: gold
+schema: “gold”
upf:
+tags: upf

tests:
caspian_financials:
pre-hook:
- “{{ generate_query_tags() }}”

macro definition:
{% macro generate_query_tags() %}
{% do log(‘Executing generate_query_tags macro’, info=True) %}
{% set airflow_dag_id = var(“airflow_dag_id”, “unknown_dag”) %}
{% set airflow_run_id = var(“airflow_run_id”, “unknown_run”) %}
{% set airflow_task_execution_date = var(“airflow_task_execution_date”, “unknown_execution_date”) %}

{% set query_tag = {
“airflow_dag_id”: airflow_dag_id,
“airflow_run_id”: airflow_run_id,
“airflow_task_execution_date”: airflow_task_execution_date
} | tojson %}

– Run the ALTER SESSION statement
{% set sql %}
ALTER SESSION SET QUERY_TAG = ‘{{ query_tag }}’;
{% endset %}

{{ return(sql) }}
{% endmacro %}

schema yml file where the tests are defined

version: 2

models:

  • name: upf_ref
    tests:
    • dbt_expectations.expect_table_row_count_to_be_between:
      min_value: 1

The pre-hook works fine when running the model but not getting triggered when the tests are getting executed.

the DAG uses DbtOperator and passes dbt_verb=“test” with vars containing airflow context