I want to generate the tables in my warehouse only after they pass the required test.
I have an automation service in place that moves files from one data warehouse location to a remote service. I am looking for a way to generate the files only after ensuring that they have been tested, rather than the other way around.
Are you doing something like an UNLOAD command which creates a Parquet file on S3?
If so, if you build your project as normal, including all the tests you need to validate the data, you could set up an on-run-end macro that issues the unload commands. In conjunction with the on-run-end context variables, you could check that the tests passed and then perform the upload if everything’s OK