Reference Model Tags in Macro

Help
, ,
1

The problem I’m having

As a dbt developer, I want to pass each Model’s Tags into a Macro so I can programmatically determine the Snowflake Warehouse for each Model.

The context of why I’m trying to do this

We have specific Model Tags throughout all of our dbt Models (e.g. “semantic_model_data”, “cf_data”). In our dbt orchestration, we are setting environmental variables including the Snowflake Warehouse to use for the dbt run command. However, we want to split out dbt Models with specific Tags into use-specific Snowflake Warehouses.

For example, the dbt run command uses DEV_WH. We have the following Models || Tags:

  • table1 || standard_data
  • table2 || cf_data
  • table3 || semantic_model_data
  • table4 || standard_data

I want the Macro logic to work as follows:
table1 and table4 should run on the default/environmental-variable-defined WH of DEV_WH.
table2 should run on WH CF_DATA_WH
table3 should run on WH SM_DATA_WH

My Macro is as follows for this logic:

{% macro set_warehouse_by_tag(tags) %}
    {% if 'cf_data' in tags %}
        {% do return('CF_DATA_WH')%}
    {% elif 'semantic_model_data' in tags %}
        {% do return('SM_DATA_WH') %}
    {% else %}
        {% do return('DEV_WH') %}
    {% endif %}
{% endmacro %}

The challenge that I have is getting the Models’ tags into the Macro.

What I’ve already tried

From How to use tags as Macro parameters, I tried using “model.tags” in my Model Config Blocks as follows:

{{config(
    materialized = 'incremental',
    enabled = true,
    unique_key='unique_key',
    tags=["dbt_run:hourly","semantic_model_data"],
    snowflake_warehouse=set_warehouse_by_tag(model.tags),
    cluster_by=['customer_schema']
)
}}

If this correctly passed the “model.tags” value to the Macro, the Macro should receive “dbt_run:hourly,semantic_model_data”, line 4 of the Macro should be true since “semantic_model_data” is in ‘Tags’, and the WH should be SM_DATA_WH for this Model. However, I’m seeing that the default DEV_WH is being used which tells me that the “model.tags” are not being passed as I expected.

I’ve tried “tags”, “node.model.tags”, I’ve tried adding another set of curly braces (even though the Config block is already in curly braces), etc. and I can’t get the Tags passed into the Macro to determine the Snowflake Warehouse.

Ultimately, I manually copied the Tags value into the Macro call, but maintaining those over 2000+ Models is a lot of effort and is a magnet for problems.

How do I accomplish this goal of setting Tag-specific Snowflake Warehouses??