Adding a column to an incremental model

Archive
1

What will dbt do if I add a new column to an existing incremental model? Add the column and keep the value for it as NULL for older rows?

1 Like
11

I’ve hidden the old discussion since it’s no longer accurate, but this page still ranks highly in search results, so a quick update:

As of v0.21 (back in October 2021!), you can configure on_schema_change behaviour, choosing between

  • ignore (default): Don’t add new columns, fail if a column has been removed
  • fail: Triggers an error message when the source and target schemas diverge
  • append_new_columns: Append new columns to the existing table. Note that this setting does not remove columns from the existing table that are not present in the new data.
  • sync_all_columns: Adds any new columns to the existing table, and removes any columns that are now missing. Note that this is inclusive of data type changes. On BigQuery, changing column types requires a full table scan; be mindful of the trade-offs when implementing.

None of these will backfill a new column’s values in old rows. In that case you will still need to --full-refresh your model (or manually update the records in a one-off operation outside of dbt).

For more details and examples, check out the docs