Hello!
Currently my team is attempting to improve our deployment process by only building models as needed with dbt based on actual changes made.
For context, we are using Github Actions to run our models regularly on a timed basis, but when we make new deployments to the application, of course we want to not have to run all of the models over again, but only they ones that have been modified.
I found this documentation, laying out how to run only the modified model using artifacts, however I have a question in terms of the practicality of this in a CI/CD environment.
Our Github Actions being stateless, I am curious how the dbt run command would actually be able to determine these artifacts. I assume I should not commit the modified artifacts to the git repo, as that would not really make sense.
Is it possible that I am looking in the wrong spot for resources on this? Has anyone figured out their own way of going about this?
Any help would be greatly appreciated.