A few beginner questions on incremental capability

  1. Incremental update does not by default support updating a subset of columns. If this is something that you think will be useful for you (your example sounds like event streaming so maybe?) it is possible to write your own materializations that support this.

  2. It is up to you to decide how the incremental update decides which rows to use. The most common example is to have a where filter that only includes rows based on a timestamp/version column being greater than anything already in the table. If you are only ever interested in inserts then you could filter on if the unique key exists in the table aready.

1 Like