Multiple dbt runs on same model but different time range create same tmp table

Help
1

The problem I’m having

I am trying to initiate multiple runs for the same dbt model each for a different time range. It returns error model1__dbt_tmp already exist for the 2nd run.
e.g.
dbt run --select model1 --vars {“min_date”: “2023-09-23T00:00:00+00:00”, “max_date”: “2023-09-24T00:00:00+00:00”
dbt run --select model1 --vars {“min_date”: “2023-09-24T00:00:00+00:00”, “max_date”: “2023-09-25T00:00:00+00:00”

The context of why I’m trying to do this

This is a snapshot type of model. I want to be able to quickly backfill past snapshot based on a changelog stream (CDC).

What I’ve already tried

Some example code or error messages

model1__dbt_tmp already exist 

dbt run --select model1 --vars {"min_date": "2023-09-23T00:00:00+00:00", "max_date": "2023-09-24T00:00:00+00:00"
dbt run --select model1 --vars {"min_date": "2023-09-24T00:00:00+00:00", "max_date": "2023-09-25T00:00:00+00:00"
2

For some materializations dbt creates temporary objects in the process. This is what you’re experiencing now. Therefore you should avoid concurrent runs of the same model. (I’m pretty sure I’ve read this before in the documentation somewhere but can’t find it at the moment)

A nasty workaround: what I’ve done in the past, but really won’t recommend is altering the materilization behaviour to create a unique tmp name.

Note: @diko originally posted this reply in Slack. It might not have transferred perfectly.

1 Like
3

Thanks for the reply!