Hello! I want to snapshot a field that results from several layers of data transformations and I’d appreciate some help thinking through whether it makes sense to build this from the source table. It would be analogous to a situation where:
- I have a field called
- That field is a combination of fields called
customer_has_reviewed_widgetfrom two different upstream tables
- Each of those fields is a combination of multiple fields from multiple upstream tables
Using the best practice of snapshotting the source tables, that means that I would have to re-transform all the data that results in
customer_prefers_widget. This is both time-consuming and creates a vulnerability if the team in future decides to change logic behind
I’m wondering if others have encountered this situation and how they’ve dealt with it?