Snapshotting transformed fields

In-Depth Discussions
,
1

Hello! I want to snapshot a field that results from several layers of data transformations and I’d appreciate some help thinking through whether it makes sense to build this from the source table. It would be analogous to a situation where:

  1. I have a field called customer_prefers_widget
  2. That field is a combination of fields called customer_has_bought_widget and customer_has_reviewed_widget from two different upstream tables
  3. Each of those fields is a combination of multiple fields from multiple upstream tables

Using the best practice of snapshotting the source tables, that means that I would have to re-transform all the data that results in customer_prefers_widget. This is both time-consuming and creates a vulnerability if the team in future decides to change logic behind customer_prefers_widget.

I’m wondering if others have encountered this situation and how they’ve dealt with it?