Data not being incremented on dbt run

[laurodelacerda]

The problem I’m having

I am having an issue getting the incremented rows when running dbt run. No rows are being incremented at all. The model only gets updated when I run the above command with the --full-refresh parameter.

The context of why I’m trying to do this

I’m running on a Windows cmd, with the command dbt run -m my_model. I’m using dbt adapter snowflake=1.7.1 .

What I’ve already tried

• I tried to union (all) two tables with the equal schema to get the new rows from one of them
• I created the model based on one table then I tried to add the other, still unsuccessful

Some example code or error messages

[a_slack_user]

What are the configuration settings for your model and what are you using inside the {% if is_incremental() %} block?

_{Note: @Owen originally posted this reply in Slack. It might not have transferred perfectly.}

[laurodelacerda]

That’s the config

{{
config(
materialized=‘incremental’,
unique_key=‘id’,
incremental_strategy=‘merge’,
pre_hook=“ALTER WAREHOUSE dbt_DEV_WH SET WAREHOUSE_SIZE = ‘Large’”,
post_hook=“ALTER WAREHOUSE dbt_DEV_WH SET WAREHOUSE_SIZE = ‘X-Small’”
)
}}

and this is the

{% if is_incremental() %}
where filename not in (select filename from {{ this }})
{% endif %}

but I already used different strategies like

{% if is_incremental() %}
where updated_at not in (select max(updated_at) from {{ this }})
{% endif %}

[laurodelacerda]

It is relevant to say that when I run the query using dbt power user, I can find the new lines.

And before this step I am running a cte with qualify clause (maybe this is irrelevant)

[the_consultant]

You are performing the incremental load based on the filename column, but are defining id as unique key. Are you sure that’s what you want?

[laurodelacerda]

I figured out what the issue was.
I changed the filename column to a date-based column.