I’m working with a wide variety of incremental models currently and while the testing of full refresh vs incremental is great locally, it’s never truly representative of what an actual ‘incremental refresh’ will look like:
While testing, I’ll frequently have 0 new records to insert, owing to the frequency of the source data refresh, and the required frequency of my local testing. However, when it comes time to run this in cloud CI, the query pulls in new data and the local incremental runtime is no longer representative of what we see in a prod incremental refresh.
Is there a way to test these incremental runs but to change the properties of the incremental portions during the test, so that the local test becomes more representative of what we would see in a prod refresh? Any and all thoughts appreciated 