Running dbt tests on sources in airflow

Help
1

Hello, i’m trying to run dbt tests on sources in airflow. What i’ve tried is using DbtTaskGroup operator but it seems like it only runs the tests on models, is there a way to also use this operator for running tests on sources?

I want to run them on sources because i don’t want fo recreate every table in warehouse as model.

2

@batman_007 . You can create generic test and attach into source table.

for Eg:

  • Create generic folder under test and write your test SQL model.
    tests/generic/data_count_check.sql

  • data_count_check.sql

       {% test data_count_check(model, threshold) %}
   
   SELECT COUNT(*) AS raw_record_count 
     FROM {{ model }}
   HAVING raw_record_count> {{threshold}}

  • schema.yml file

            version: 2
        
        sources:
          - name: raw_customer
            tables:
              - name: customer
                tests:
                  - data_count_check:
                      tags:
                        - data_count_dq_check
                      severity: warn
                      threshold: 5

  • dbt test --select tag:data_count_dq_check

Hope this helps !!