Hi DBT team,
I am trying to build an incremental model for my project and I want to update all except 2 columns ( created_date and created_by (to preserve history)) in my target table during merge update part of incremental strategy.
Below is what I am trying to do,
{%- set columns = adapter.get_columns_in_relation(ref(‘my_table’))|map(attribute=‘column’)|map(‘lower’)|list -%}
{%- set filtered_columns = columns|reject(“equalto”,“created_by”)|reject(“equalto”,“created_date”) -%}
{%- set merge_columns = filtered_columns|join(“‘,’”) -%}
{{
config(
materialized=‘incremental’,
unique_key=‘my_key’,
merge_update_columns = [merge_columns],
tags=[“derived”]
)
}}
select * from my_table
However, this "merge_columns value is set to NULL ( blank) in the config block as I could see the merge statement with BLANK for columns in when matched update block
when matched then update set
= DBT_INTERNAL_SOURCE.
Could you please help me get this resolved?.
When I convert to merge_update_columns = [var(merge_columns)], the dbt run still fails but I could see all columns listed using my macro. It fails for variable ( list of columns) not defined error which is expected.
I am also interested to know if there is a way I can implement "when matched and (src.colA <>tgt.colA) then update set tgt.colA=src.colA " in DBT
Thanks
Shimith PM