Configuration Change for Incremental Model on DBT

Help
1

On one our previously created incremental models, I added partition_by and partition_expiration_days parameters to the configuration to set the table partition and retention in place.

{{ config(
        materialized='incremental',
        unique_key='record_id',
        on_schema_change='append_new_columns',
        partition_by={
              "field": "row_ts",
              "data_type": "timestamp",
              "granularity": "day"
        },
        partition_expiration_days = 365
        )
}}

I observed on the next run that the configuration didn’t applied to the table.

It seems a full-refresh operation needed here. Yet we have strict retention on the data source for this table which some of the data would be lost with full-refresh operation.

Could anyone please let me know how this issue can be addressed with a solution?

2

Hi guz!
I would clone that table ->clone_table
Delete the original table
Insert the clone_table aterialized=‘table’ in the ordinal_table
and the activate the incremental model again

1 Like
3

Is it one time activity?

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.