Intro
There are some excellent articles floating around describing tidy ways to organize your dbt project. Of course, these are mostly guidelines, and you will likely want to tweak or expand upon them to meet the specific needs of your project. As your project and project team grow and develop, you may find it difficult to enforce the organizing principles of your project, even if you’ve got stellar documentation and a healthy code review process. I developed dagrules to provide a structured way to codify your organizing principles and provide a mechanism to automatically check that your project is conforming to those principles.
Example
To demonstrate how dagrules works, we’ll take an example from the how we structure our dbt projects article. Suppose we want to enforce that all “staging” models must be named with the prefix “stg_” and that all “dimension” models can only depend on “staging” or “intermediate” models. To do so, we would write a dagrules.yml file like the following:
---
version: '1'
rules:
- name: Non-base staging models must be prefixed with stg_
subject:
tags:
- include: staging
exclude: base
must:
match-name: /stg_.*/
- name: Dimension models must only depend on staging or intermediate
subject:
tags:
include: dim
must:
have-parent-relationship:
require-tags-any:
- staging
- intermediate
This file should be placed in your dbt project folder and can be run against your compiled dbt project by running dagrules --check. For more details, see the dagrules project.
Acknowledgements
dagrules was originally inspired by the Oliver Twist project, which also provides an mechanism to audit your dbt dag, but I found that I wanted more flexibility in the kinds of rules that I could enforce in my own project. Big thanks to the Oliver Twist team!