Snapshots rebuilt in every run

Help
1

The problem I’m having

Every time I run dbt build or dbt snapshot, the snapshots are rebuilt. It isn’t checking the previous data, only inserting the most recent data with new dates.

The context of why I’m trying to do this

The source for the snapshot is a table built in the same model to check inconsistencies on the data, my goal is to check when those inconsistencies change.
Table:

id dimension issue
1 uniqueness duplicated product
3 uniqueness duplicated product
2 accuracy wrong category

I am expecting that when some of the issue gets solved, the dbt_valid_to column of the snapshot table be populated. But currently the snapshot is being rebuilt even without any change in the source table.

What I’ve already tried

I already tried only one or two columns in check_cols, without the invalidate_hard_deletes.
I already changed the query.
I already tried to manually change one row and run snapshot, but all the table is rebuilt.
I already tried the timestamp strategy.

Some example code or error messages

    {{
        config(
            target_schema=generate_schema_name("DQ"),
            unique_key='id',
            strategy="check",
            invalidate_hard_deletes=True,
            check_cols="all",
        )
    }}

The compiled code doesn’t have any merge/insert statement, only create

1 Like
2

Hi @fije_red ,
I am facing same issue, were you able to find solution for this.

Regards
Nitin

3

I found the solution after checking the slack community.
The issue was in the macros generate_schema_name, I tested with the final name of the schema instead of the macro and it worked.
Then I realized that were missing some “-” in my macros, so…
fom this:

        {% endif -%}

    {% endif -%}

{% endmacro -%}

I ended up with this and it’s working well:

        {%- endif -%}

    {%- endif -%}

{%- endmacro -%}
1 Like