I’m working on a dbt project with Postgres database, currently there are ~40 models there and they all run fine if I select them with --models selector, however when I try to run all models with dbt run one of the models (relatively large/complex in the context of the project - a few millions of rows, generates in ~5 mins) gets stuck and does not finishes. Oddly enough I can observe Postgres reporting 3 identical CREATE TABLE queries being executed, however no locks are reported by Postgres.
How many threads are you using? (See understanding threads for background)
Do you see the same issue if you try doing dbt run --threads 1? If you’re currently using many threads (say, 8) and it works with a single thread, you could try changing down the number of threads to trade off parallelisation and reliability.
Thanks, seems to solve the issue, I was running DBT with 2 threads, as Postgres instance has 2 vCPUs, but something was obviously deadlocking after all.
1 Like
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.