is_incremental not triggering upstream from incremental model

Help
1

Hello everyone,
I’m new to this forum. I took over a relatively complex dbt project where there are a lot of transformations going on.

The problem I’m having

I set up incremental refresh on the main fact table and I want to do the filtering by timestamp upstream where I load in the raw data. However the macro is_incremental is not triggering upstream.

The context of why I’m trying to do this

I"m following the advice on: Configure incremental models | dbt Developer Hub
“In some warehouses, filtering your records early can vastly improve the run time of your query!”

What I’ve already tried

I set the below code in both the incremental model and the upstream model where I want to filter. The Snowflake query history shows that the test comment is only visible for the incremental model but not for the upstream model where I want to to the filtering.
{% if is_incremental() %}
– test is_incremental
{% endif %}

Please help. :slight_smile:

Thanks and best regards,
Richard

2

Hey @richard, have you made sure to go through the checklist for the macro triggering as true?

  • The model must already exist in the database
  • The destination table already exists in the database
  • The full-refresh flag is not passed
  • The running model is configured with materialized='incremental'

The fact that your upstream model isn’t showing the comment suggests that one of these might not be satisfied. Particularly materiazlied='incremental' to override any model settings you might have defined elsewhere in a schema.yml/project.yml file.

3

Hi @DataNath,
The problem is that checklist item 4 is not fulfilled because I want to do the filtering upstream of the incremental model.

I worked around the issue by using variables:
{% if var(‘is_incremental_custom’, false) %}
…
{% endif %}

And then for the incremental refresh runs building the project like this: dbt build --vars ‘{is_incremental_custom: true}’

If you have a better solution please let me know.

Thanks and best regards,
Richard