We are trying to migrate tables from mssql into snowflake, some of these tables are dimensions tables from the source tables, with columns similar to the ones dbt snapshots have (dbt_VALID_FROM, dbt_VALID_TO, dbt_SCD_ID).
We have started using dbt to get the data from snowflake, create models to do some transformations and use snapshots. The problem we have is that we want to create snapshots for the source table and then continue using the snapshots with the data from snowflake, so we keep the history we had.
We have created staging files to get the data from the source tables, then created models to do the transformations and finally the snapshots. But the problem is that we get duplicate column name ‘dbt_VALID_FROM’ when creating the snapshot from the dimension table. Is there a way to initialized a snapshot using data from a history table and replacing the columns dbt_VALID_FROM, dbt_VALID_TO, dbt_SCD_ID with the ones we have (keeping the dbt column names)?
Thanks in advance!!