Database error "cannot start commands during a parallel operation" with dbt-postgres

Help
,
1

I’m getting the following message when trying to execute dbt run locally (this error message has been shortened for brevity, but the full output is below):

Database Error in model ... cannot start commands during a parallel operation

I’m on dbt-core=1.8.3, dbt-postgres=1.8.2, and working with a Postgres 16.3 instance hosted by Tembo. All models in my project are materialized as tables.

The compiled SQL code for the two staging models that are erroring out runs just fine through psql, so it seems like the error has something to do with dbt. Additionally, I get the same error when I try to run either of the two failing models individually (i.e., dbt run --models scorecards or dbt run --models scorecard_entries). All 3 of the staging models are unpacking and casting fields from jsonb columns in raw source tables - no joins or other complex transformations.

Full output

22:31:54  Running with dbt=1.8.3
22:31:55  Registered adapter: postgres=1.8.2
22:31:55  Found 6 models, 3 sources, 415 macros
22:31:55  
22:31:57  Concurrency: 1 threads (target='dev')
22:31:57  
22:31:57  1 of 6 START sql table model dev_stage.courses ................................. [RUN]
22:32:22  1 of 6 OK created sql table model dev_stage.courses ............................ [SELECT 29302 in 24.68s]
22:32:22  2 of 6 START sql table model dev_stage.scorecard_entries ....................... [RUN]
22:32:22  2 of 6 ERROR creating sql table model dev_stage.scorecard_entries .............. [ERROR in 0.36s]
22:32:22  3 of 6 START sql table model dev_stage.scorecards .............................. [RUN]
22:32:23  3 of 6 ERROR creating sql table model dev_stage.scorecards ..................... [ERROR in 0.32s]
22:32:23  4 of 6 SKIP relation dev_intermediate.scorecard_entries_exploded ............... [SKIP]
22:32:23  5 of 6 SKIP relation dev_intermediate.scorecards_exploded ...................... [SKIP]
22:32:23  6 of 6 SKIP relation dev_analytics.hole_scores ................................. [SKIP]
22:32:23  
22:32:23  Finished running 6 table models in 0 hours 0 minutes and 27.89 seconds (27.89s).
22:32:23  
22:32:23  Completed with 2 errors and 0 warnings:
22:32:23  
22:32:23    Database Error in model scorecard_entries (models/stage/scorecard_entries.sql)
  cannot start commands during a parallel operation
  compiled Code at target/run/udisc_dbt/models/stage/scorecard_entries.sql
22:32:23  
22:32:23    Database Error in model scorecards (models/stage/scorecards.sql)
  cannot start commands during a parallel operation
  compiled Code at target/run/udisc_dbt/models/stage/scorecards.sql
22:32:23  
22:32:23  Done. PASS=1 WARN=0 ERROR=2 SKIP=3 TOTAL=6

What I’ve already tried

  • The standard dbt uninstall and reinstall and full machine restart
  • Have looked to see if there are long-running dbt jobs locally or on our Postgres instance and can’t find evidence of either
  • We set the database default_table_access_method to ‘columnar’ (using the Hydra columnar extension), but I receive the same error message if I reset it to ‘heap’
  • I’ve deleted the compiled source code in the target/ folder and re-generated it, but still get the error

Has anyone encountered this error before? My only hunch so far is that I was able to successfully run these dbt models on an instance with Postgres 15.3 and haven’t been able to since we upgraded it to 16.3. I’d prefer to figure out a solution that works with this Postgres version and not downgrade because that may break other pieces of our infrastructure.

2

Same issue here

Note: @Ahmed A. originally posted this reply in Slack. It might not have transferred perfectly.

3

This issue has persisted but it seems like it is more related to Postgres and/or the Hydra columnar extension in Postgres. I’m going to consider this issue closed in dbt but if anyone encounters the same issue and wants to follow up on the tickets submitted to those platforms:

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.