DBT IDE populates the data in table but same command executed from DBT cloud job does not.

In-Depth Discussions
, ,
1

Hello,

When I execute the dbt run command from the dbt IDE, the table populates with data. However, when I run the same command from a dbt cloud job, the data does not populate.
Here’s my setup:

  • I’m using the incremental strategy and the append method for models.
  • I’ve added a macro in the pre_hook to truncate the table before loading data. This helps resolve access issues since dbt drops the tables for models they don’t have any strategy.
  • Initially, I only used the macro, but dbt build didn’t populate data when executed from dbt cloud jobs.
  • When I call this macro under the before_begin macro, dbt cloud jobs start populating data in the table.
    I’m curious about the reason behind this behavior. Why does using the before_begin macro allow dbt cloud jobs to populate the data successfully?

dbt models configuration.

{{
config(
materialized = ‘incremental’,
incremental_strategy = ‘append’,
pre_hook= [
before_begin(
audit_entry(‘hcp_ok_universe’,)
),
before_begin(“{{truncate_table_if_exists()}}”)
]
)
}}

select * from stg_models