I have an incremental model and I want to exclude a few columns (autoincrement columns in Snowflake as well columns with default value as CREATED_ON col) For this I use merge_exclude_columns (as described in the doc (Incremental models | dbt Developer Hub).
So my config is:
{{ config(materialized=‘incremental’,
alias= var(‘tgt_tbl’) ,
unique_key= var(‘ukey_nm’),
incremental_strategy= var(‘incrtype’),
merge_exclude_columns = [‘TIME_CREATED_ON_DATE’,‘STORE_CODE_AUTOKEY’,‘PRODUCT_AUTOKEY’],
schema=var(‘tgt_schm_nm’),
transient=false,
pre_hook= [use_warehouse(var(‘vwh’))],
)
}}
The dbt command with the variables looks like:
dbt run -s cdf_rpos_mod_incremental --vars ‘{vwh: MY_WH, tgt_schm_nm: MY_SCH,tgt_tbl_nm: TARGET_TAB,load_id: 22,ukey_nm: PRIMARYKEY_HASH,incr_col_nm: ,incrtype: merge}’
The problem I’m having that the unwanted columns are not present in the merge branch but the insert branch still has the columns that should be excluded:
merge into MY_DB.MY_SCH.TARGET_TAB as DBT_INTERNAL_DEST using MY_DB.MY_SCH.TARGET_TAB__dbt_tmp as DBT_INTERNAL_SOURCE on DBT_INTERNAL_SOURCE."PRIMARYKEY_HASH" = DBT_INTERNAL_DEST."PRIMARYKEY_HASH" when matched then update set "STORE_JSON" = DBT_INTERNAL_SOURCE."STORE_JSON", … ,"PRIMARYKEY_HASH" = DBT_INTERNAL_SOURCE."PRIMARYKEY_HASH" when not matched then insert ("STORE_JSON", …, "PRIMARYKEY_HASH", "TIME_CREATED_ON_DATE", "STORE_CODE_AUTOKEY") values ("STORE_JSON", …, "PRIMARYKEY_HASH", "TIME_CREATED_ON_DATE", "STORE_CODE_AUTOKEY")
Anyway to exclude the columns completely from the statement?