Hey guys! I am pretty excited about DBT.
I cannot similar questions so bare with me here.
Does incremental function allow update only selected column? For example, I want the update only happens to last_updated_at but not first_received_at column if the unique key match.
Can we skip the update if the unique key shows the record already exit in the table? simple example, my table has user_id and session_id mapping and I only want the new unmapped record to be added in there.
Incremental update does not by default support updating a subset of columns. If this is something that you think will be useful for you (your example sounds like event streaming so maybe?) it is possible to write your own materializations that support this.
It is up to you to decide how the incremental update decides which rows to use. The most common example is to have a where filter that only includes rows based on a timestamp/version column being greater than anything already in the table. If you are only ever interested in inserts then you could filter on if the unique key exists in the table aready.
It’s possible to make both of those things happen with dbt incremental models—the trick is in how you write the code. You control what updated records get written to the table, so you can always express things like only include updates if the unique key does not yet exist just by expressing them in straightforward SQL. And if you want your first_received_at column to always stay the same, just make sure the SQL that defines that column is always doing a min()!
Thanks ! This is really helpful