The problem I’m having
I can’t create documentation programmatically or using macros.
The context of why I’m trying to do this
Hello everyone ! I’m a new user of dbt and I’m currently using macros to create new columns on a table. However, because I’m connected to Snowflake, I would love to make documentation for those new columns. I know you can make descriptions in the .yml file corresponding to the table but I want to be able to “fill” programmaticaly those descriptions.
For example, if i’m using the macro (example below) that checks if a row is formatted by a regex line on a column named a, it will create a column named “a_regex”. I would like my macro to add on the .yml file the description ‘Column that checks if “a” is correctly formatted’.
What I’ve already tried
I already tried some code using the COMMENT function used by Snowflake which is not known by dbt leading to an error during compilation and I already searched if by any means you could execute shell script using macros but couldn’t find anything. I also saw the manifest.json that seems to be the file sent to Snowflake with the documentation but I don’t know how I can modify it programmatically before sending if possible.
Some example code or error messages
{% macro regex_email(column_name) %}
{% set regex_column = column_name ~ '_QUAL_IS_REGEX_EMAIL' %}
{% set regex_pattern = '^[\\\\w\\\\-\\\\.]+@([\\\\w\\\\-]+\\\\.)+[\\\\w\\\\-]{2,4}$' %}
case
when REGEXP_LIKE({{ column_name }}, '{{ regex_pattern }}') then 1
else 0
end as {{ regex_column }}
--TODO: Add documentation for the new column like "Column that checks if the column {{column_name}} is correct using the following pattern : {{regex_pattern}}"
{% endmacro %}
Thanks a lot for your help if you already encountered a similar issue !