The problem I’m having
How can we control updates of the incremental model to ensure that any changes in data type in individual fields are not automatically propagated to the incremental model?
The context of why I’m trying to do this
We have an incremental model that gets 100+ million new items every day. If someone for any reason changed the length of a varchar field in this model, we do not want to automatically extend this field when the model is run.
It would be better to do this change manually to have control over the situation.
What I’ve already tried
We tried to set the “on_schema_change” value to “fail”, but that does not stop the update of the data type of individual fields.
Is the only way to do this to implement a contract for this model to keep the data types avoiding any change by mistake?
The documentation is not clear that the “on_schema_change” only applies to new/deleted columns and not to the data type on individual columns.