tags in dbt to be applied on databricks tables/views etc

Help
1

We have

Tags in our models and want this to reflected in Databricks Tags for tables /views.
Anyone manage to accomplish this?

2

Hey @ThuanNg

I came up with a macro to it, that you can use a Post Hook. You can only apply either groups of keys, or key=values to Databricks as tags, (they can’t be mixed), so I made it that by default only key=values are applied. This means you can keep the normal key like tags for dbt use.

If you don’t want the KW tags, then set key_values to false, and key only tags will be applied.

macros/apply_table_tags.sql

{% macro apply_table_tags(key_values=true) %}
{%- set tags_list = [] -%}
{%- for tag in model.config.tags -%}
    {%- if key_values is true and '=' in tag -%}
        {% do tags_list.append(tag) %}
    {%- elif key_values is false and '=' not in tag -%}
        {% do tags_list.append(tag) %}
    {%- endif -%}
{%- endfor -%}

{%- set tags_string -%}
   {{ '"' + tags_list | join('","') | replace('=', '"="') + '"' }}
{%- endset -%}

{%- set comment_query -%}
    alter table {{ this | lower }} set tags ({{ tags_string }})
{%- endset -%}

{%- if tags_list | length > 0 -%}
    {{ return(comment_query) }}
{%- endif -%}

{% endmacro %}

dbt_project.yaml

model:
  <resource-path>:
    +post-hook:
      - "{{ apply_table_tags(key_values=true) }}"
3

Today I also came up with a small variation of the Post-Hook macro, where the Meta config is used instead for Tags in Databricks because it’s inherently key value. This will allow the tags in dbt to be used for normal use like query selection.

macros/apply_table_meta.sql

{% macro apply_table_meta() %}
{%- set tags_list = [] -%}
{%- for item in model.config.meta.items() -%}
    {% do tags_list.append(item | join('"="')) %}
{%- endfor -%}

{%- set tags_string -%}
   {{ '"' + tags_list | join('","') + '"' }}
{%- endset -%}

{%- set comment_query -%}
    alter table {{ this | lower }} set tags ({{ tags_string }})
{%- endset -%}

{%- if tags_list | length > 0 -%}
    {{ return(comment_query) }}
{%- endif -%}

{% endmacro %}

dbt_project.yaml

model:
  <resource-path>:
    +post-hook:
      - "{{ apply_table_meta() }}"
4

Great. do you have a sample of of a model showcasing the tags format?