How to import variable (i.e. {%set foo=bar%}) from another file? No loader for this environment specified

Help
,
1

I have a variable that I need to re-use in different files, and I have trouble importing it. E.g. several hundred lines of:

{% set auto_curated_columns = { “oldname”:“newname”, “old_col_2”:“new_col_2”} %}

I would like to do
{% from ‘file_with_definition’ import auto_curated_columns %}
but that errors out with: “no loader for this environment specified”.

Vars defined in dbt_project.yaml is not a good fit extensive implementation details should really not go there.

It wouldn’t even fit into an environment variable.

A template would not cut it - I am using the variable in loops, conditionals, to derive code, to derive column names such as import_status_of_{{auto_curated_columns.loop}}, etc. (Sticking the file with the definitions into the macros directory doesn’t work either.)

For now, I’m stuck pasting the variable definition into tests and yaml-files. At least for models, I can define vars:auto_curated_columns in dbt_project.yml and then {{ set auto_curated_columns = fromjson(var(‘auto_curated_columns’))}}.

How do I import a variable directly, instead?
How do I use Jinja in schema.yml?

2 Likes
2

Were you able to find a solution to this? I also need to do exactly the same.

3

I ended up defining lots of variables in dbt_project.yml because that is the only spot that is imported into multiple files.

4

You can create a macro which returns a dictionary generated from yaml set as a variable within the macro. e.g.

{% macro load_curated_columns() %}
{% set yml_str %}

auto_curated_columns:
  old_col_1: new_col_1
  old_col_2: new_col_2

{% endset %}
{% set conf_yml = fromyaml(yml_str) %}
{{ return(conf_yml) }}
{% endmacro %}

Then, when you need to load these into your model or test, you can

{% set auto_curated_columns = load_curated_columns()['auto_curated_columns'] %}
select
    {{ auto_curated_columns.old_col_1 }} as {{ auto_curated_columns.new_col_1 }}
    {{ auto_curated_columns.old_col_2 }} as {{ auto_curated_columns.new_col_2 }}
from old_table

or whatever.

1 Like
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.