I notice in the docs that it says:
exclude … is always applied last within its scope.
Let’s say I have the following lineage for an output model used by a Tableau report:
inventory --------------------->
int_products_enhanced --------->
--> rpt_tableau
sales ------------------------->
And the following lineage used by my sales model:
int_products_enhanced ----->
stg_brick_mortar_sales ---->
---> sales
stg_ecommerce_sales ------->
You’ll notice that int_products_enhanced
is used in both transformations.
The sales
model, along with all it’s parents, is a heavy model I want to exclude from hourly runs (because I run the sales
model daily).
That said, because the lineage for the rpt_tableau
model also directly references int_products_enhanced
, I do want int_products_enhanced
to be re-run when rpt_tableau
and it’s parents are run.
Given that exclude selectors apply last and the following selectors.yml, int_products_enhanced
doesn’t get re-run. Suggestions on how I could alter it so that it does get re-run due to it’s being directly referenced by rpt_tableau
?
selectors:
- name: reporting_layer_refresh_excluding_daily_and_staging
definition:
union:
- method: fqn
value: rpt_tableau
parents: true
children: false
- exclude:
- method: fqn
value: sales
parents: true
children: false