The problem I’m having
Hi,
I’m trying to dynamically build the columns in the “SELECT” query utilizing a customer macro and the model’s YML file. And I’m getting ‘macro’ undefined error.
The context of why I’m trying to do this
This gives the users to control the value for the column that they want. Even though we declare a standard, that might be satisfactory to the client and this enables them to provide overwrites to a specific column.
What I’ve already tried
Desired YML Structure that the user can provide overwrites to:
version: 2
models:
- name: table_name
description: ""
tests:
- model_has_rows:
columns:
- name: COLUMN1
data_type: varchar
description: ''
- name: COLUMN2
data_type: number
description: ''
overwrite: "{{ overwrite_macro_name() }}"
Macro that defines the standard structure of the model:
{% macro gen_stage_table_name(key) %}
{# This macro generates the SQL for the table_name stage model #}
{# Check if the key is 'columns' to return the column definitions #}
{% if key=='columns'%}
{%- set product_cols =
{
"COLUMN1": "Data",
"COLUMN2": "Data"
}
-%}
{{ return(product_cols) }}
{% endif %}
{% endmacro %}
Macro that compares the column values:
{% macro gen_product_columns(macro_name, supplied_cols) %}
{%- set filtered_cols = {} -%}
{% set macro_call = "gen_stage_" ~ macro_name %}
{% set macro_to_call = context[macro_call] %}
{%- set product_cols = macro_to_call('columns') -%}
{# Iterate through supplied_cols and filter based on product_cols #}
{%- for key, value in supplied_cols.items() -%}
{%- if key in product_cols -%}
{%- if value.get('overwrite') %} {# Use the overwrite value if provided #}
{% if value.get('macro') %}
{%- do filtered_cols.update({key: context.render(value['overwrite'])}) -%}
{% else %}
{%- do filtered_cols.update({key: (value['overwrite'])}) -%}
{% endif %}
{%- else %} {# Use the product column definition #}
{%- do filtered_cols.update({key: context.render(product_cols[key])}) -%}
{%- endif %}
{%- else %} {# If the key is not in product_cols, use the supplied value #}
{%- do filtered_cols.update({key: context.render(value['overwrite'])}) -%}
{%- endif %}
{%- endfor -%}
{%- for key, value in filtered_cols.items() %}
{{ value }} AS {{ key }}{%- if not loop.last -%},{%- endif -%}
{%- endfor -%}
{% endmacro %}
Final Model File:
SELECT DISTINCT
{{gen_product_columns("table_name", model.columns)}}
FROM SOURCE
Some example code or error messages
Compilation Error
Could not render {{ overwrite_macro_name() }}: 'overwrite_macro_name' is undefined