Great question. dbt is “stateless”—it has no idea whether a model is been run before, if a model has changed, if a file has been edited, nothing. As such, when you do a dbt run, dbt simply generates SQL based on the current code in your project. And the code for an incremental model, essentially, does the following:
- generate a temp table with the contents of your model select statement
- delete rows from your existing table with unique keys that match keys that exist in your temp table
- insert all records from your temp table into the target table
- drop the temp table
The problem comes when you hit step #3. If the columns in your temp table are not the same as the columns in your target table, the insert will likely fail. This is one of the reasons the --full-refresh flag exists (docs here)! When you make changes to your incremental models, you need to to a --full-refresh run and dbt will rebuild all of your incremental models from scratch using the current code.