How to work with tables managed by ORMs

We have the following pipeline

  1. Raw data is ingested by Airbyte into a staging database
  2. Some transformation and aggregation are done by DBT
  3. Records from 2 is inserted into a destination table to be consumed by a downstream app, which is a web application. All older data in the table is cleared before the insert (a transaction is needed since rollback of the deletion of older data is required if the insert of new fails).

Destination table in step 3 is created and managed by the ORM framework (SQLAlachemy) of the web app.


  1. What materialization strategy shall I use? Since the table in 3 is managed externally, I do not want DBT to drop/create the table every time it runs. So, I assume incremental materialization is a good choice.
  2. How to clear the older data before DBT inserts the new records? I assume a hook might be a good idea. Shall I use pre or post hook?
  3. How to handle the auto primary key during insert? The table already contains an auto generated primary key. Therefore, during insert, the PK column is not needed. How to tell DBT to control the insert statement on a finer granular level?
  4. How to enable transaction during the insert? Is this by default enabled in incremental materialization?