The problem I’m having
I am trying to migrate code from at stored procedure in T-SQL to an incremental dbt model.
In the stored procedure, we are using
OPTION (USE HINT (‘ENABLE_QUERY_OPTIMIZER_HOTFIXES’))
in order to achieve good performance.
It seems I cannot include this code line in the dbt -model - it fails with error message “Encountered unknown tag ‘OPTION’.”.
Is there a way to solve this? Some macro that allows me to enable the query optimizer hotfixes, or some other workaround?
The context of why I’m trying to do this
We have a SQL Server database, and we use Polybase to connect to our source data, which are located in an Oracle database, through external tables.
Without the query optimizer hotfixes enabled, we get problems with the performance because the filter clause, i.e.
where date_day >= (select coalesce(max(date_day), ‘1900-01-01’) from {{ this }})
is not pushed down to the source database.
What I’ve already tried
I have tried to omit the ENABLE_QUERY_OPTIMIZER_HOTFIXES, to see if dbt somehow handles this in some other smart way. But the result is that the performance for an incremental load is like the performance for a full refresh of the model.