My team is using Fivetran and DBT to load/transform the data into Snowflake and we’re working on using incremental models.
- The initial load of the data works fine
- if there IS data to merge into Snowflake works fine
- When there ISN’T any new data to merge then we get an error in Fivetran similar to below:
[0m20:49:18 Database Error in model develop_test_model (models/develop/misc/develop_test_model.sql) e[0m20:49:18 001003 (42000): SQL compilation error: e[0m20:49:18 syntax error line 19 at position 6 unexpected ')'.
Looking into the Snowflake query history it eventually errors on a query like this (please ignore line positions, i removed whitespaces):
create or replace temporary table WAREHOUSE.DATABASE.develop_test_model__dbt_tmp as ( );
Is there a way to skip the incremental load process if there is no data to merge in?
Any help would be appreciated!