We have a use case of triggering a dbt job which builds some models only if the sources tables have data for a day. The source tables are loaded through a different ETL tool between 12 am to 10 am.
We wrote a macro which checks the source tables every 1 hour to check the count and if count = 0 we are raising an exception and the next build command is not getting called.
Although we are able to achieve what we need, its causing the dbt job to fail 10 times and only 1 success. The dbt run history and splunk dashboard is showing up failures which is not actual a fail.
How can this be scenario be achieved without making the actual job to fail.