I’m trying to develop some tests/macros that can be specified in the project models .yml file. But dbt either returns a warning that there’s nothing to do or that the selection criteria does not match any nodes.
Our source data is the culprit of most of our data issues. Identifying issues there is ideal. Although, our stage tables are just unions of the various source tables; testing their will suffice. But for my knowledge I’d like to know how to test on sources.
I started with the basic built-in tests like
unique on the models, then applied them to the sources. Running
dbt test does not work, but that might be expected behavior if the sources are not in the DAG (ie called by
source())? Some of the sources are in the DAG while others are not.
dbt test --select source:<source> results in the same issue, and actually
source:* throws a zsh error.
So then I tried adding the sources as a model and attempted to test them as such. Same issue as above.
Obviously this does not get to the heart of the issue, that I’m trying to develop my own functions, but using a built-in test is a good starting point. One question related to the real issue: do all dbt tests consider having any rows in the final query a failure? Is there a way to specify a failure using a jinja variable, maybe a boolean?
Testing as a source:
sources: - name: nm_trace description: Trace tables in snowflake tables: - name: bmsi_customers_raw columns: - name: customerid tests: - not_null
dbt test -s source:bmsi_customers_raw
The selection criterion 'source:bmsi_customers_raw' does not match any nodes [WARNING]: Nothing to do. Try checking your model configs and model specific
Running it as a model:
models: - name: bmsi_customers_raw columns: - name: customerid tests: - not_null
dbt test -s bmsi_customers_raw
[WARNING]: Did not find matching node for patch with name 'bmsi_customers_ra w' in the 'models' section of file 'models/bt_backend_data.yml' [WARNING]: Nothing to do. Try checking your model configs and model specific ation args