Model not found when passing the list of models via macro

Help
, ,
1

The problem I’m having

I am using a macro to get a list of models that have a certain tag:

{% macro get_models_by_tag(tag) %}

{% if execute %}
{% set models_with_tag = [] %}
{% for model in graph.nodes.values() | selectattr("resource_type", "equalto", "model") %}

    {% if tag in model.config.tags %}
        {{ models_with_tag.append(model.name) }}
    {% endif %}

{% endfor %}

{{ return(models_with_tag) }}

{% endif %}

{% endmacro %}

macro borrowed from this topic: Getting "'dict object' has no attribute 'nodes'" error trying to use graph.nodes.values()

But when I try to use this macro in a different model

{% set models = get_models_by_tag(tag='my_tag') %}


{% for model in models %}
    select * from {{ ref( model ) }}
    {% if not loop.last %} union all {% endif %}
{% endfor %}

I get:

 Model 'xyz' (models/xyz/xyz.sql) depends on a node named '
  ' which was not found

What I’ve already tried

I tried encapsulating my model with {% if execute %} {% endif %} and then I get a different error:

Compilation Error in model xyz (models/xyz/xyz.sql)
  dbt was unable to infer all dependencies for the model "xyz".
  This typically happens when ref() is placed within a conditional block.
  
  To fix this, add the following hint to the top of the model "xyz":
  
  -- depends_on: {{ ref('zyx') }}

When I add the line it suggests, then it works, but that’s not really what I am looking for, as I am going to have a lot of models, and I don’t really want to have a lot of depends_on lines.

Please let me know if I am missing something, can this be done without depends_on line?

2

Hi Micaga! I’ve run into this before as well, unfortunately I don’t believe there are reliable workarounds for this yet. You can see what people have tried here.

The only other option I have that may work, may not, is to do something similar to how the dbt_evaluator_package populates its base models. Which is to say, the model itself will build with zero rows, establishing the columns. Then there will be a post-hook macro that actually populates the table with the rows. This SQL since it belongs strictly in a macro would not error out. However, your lineage will look way off and this isn’t really considered best practice.
Since tags aren’t exactly dynamic (meaning they require a pull request to apply them), I would say your best bet is to have an inline jinja variable that you update with a new table each time a table is added that should be unioned.