custom test not recognized

Help
,
1

The problem I’m having

my custom test is not being recognized

The dbt project yaml file sets the tests location to this folder test-paths: ["tests"].

This is a singular test and not a generic one and therefore per dbt’s docs should not be in the generic sub-directory though I tried that anyway.

Test lives here: tests/flow_1_is_pos_or_null.sql

It is not being recognized when I run dbt test --select source:clearinghouse.STATION_RAW

The context of why I’m trying to do this

for my job

What I’ve already tried

  • moving it to tests/generic/flow_1_is_pos_or_null.sql
  • moving it to macros/flow_1_is_pos_or_null.sql
  • moving and renaming it to tests/generic/test_flow_1_is_pos_or_null.sql
  • renaming it to tests/test_flow_1_is_pos_or_null.sql

Some example code or error messages

for my file tests/flow_1_is_pos_or_null.sql

select FLOW_1
from {{ source('clearinghouse', 'STATION_RAW' ) }}
where FLOW_1 >= 0 or FLOW_1 is null

for my YAML _sources.yml

version: 2

sources:
  - name: clearinghouse
    database: RAW_PRD
    schema: CLEARINGHOUSE
    description: tk
    tables:
      - name: STATION_META
        description: tk
        columns:
          - name: FILENAME
          - name: FLOW_1
            tests:
              - flow_1_is_pos_or_null
          - name: etc...

I get two failures as seen in the screenshot. The messages are as follows:

flow_1_is_pos_or_null

02:14:46 Began running node test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null
02:14:46 1 of 2 START test flow_1_is_pos_or_null ........................................ [RUN]
02:14:46 Began compiling node test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null
02:14:46 Writing injected SQL for node "test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null"
02:14:46 Timing info for test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null (compile): 02:14:46.266582 => 02:14:46.296163
02:14:46 Began executing node test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null
02:14:46 Writing runtime sql for node "test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null"
02:14:46 Using snowflake connection "test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null"
02:14:46 On test.caldata_mdsa_caltrans_pems.flow_1_is_pos_or_null: select
      count(*) as failures,
      count(*) != 0 as should_warn,
      count(*) != 0 as should_error
    from (
      select FLOW_1
from RAW_PRD.CLEARINGHOUSE.STATION_RAW
where FLOW_1 >= 0 or FLOW_1 is null
      
    ) dbt_internal_test
02:14:46 Opening a new connection, currently in state closed
02:15:11 Snowflake adapter: Snowflake query id: 01b255e6-0002-1a56-003e-3887001e6012

source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1

02:14:46 Began running node test.caldata_mdsa_caltrans_pems.source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1.9426edd3a7
02:14:46 2 of 2 START test source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1  [RUN]
02:14:46 Acquiring new snowflake connection 'test.caldata_mdsa_caltrans_pems.source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1.9426edd3a7'
02:14:46 Began compiling node test.caldata_mdsa_caltrans_pems.source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1.9426edd3a7
02:14:46 Timing info for test.caldata_mdsa_caltrans_pems.source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1.9426edd3a7 (compile): 02:14:46.274731 => 02:14:46.283546
02:14:46 Compilation Error in test source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1 (models/_sources.yml)
  'test_flow_1_is_pos_or_null' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".
02:14:46 2 of 2 ERROR source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1 ..... [ERROR in 0.02s]
02:14:46 Finished running node test.caldata_mdsa_caltrans_pems.source_flow_1_is_pos_or_null_clearinghouse_STATION_RAW_FLOW_1.9426edd3a7
2

Hi @britt.allen! The error is caused by the added test config in the source.yml file. STATION_RAW will automatically be tested when running dbt test --select source:clearinghouse.STATION_RAWbecause the test flow_1_is_pos_or_null.sql uses a source()function. Can you try removing that from the config and then re-testing?

version: 2

sources:
  - name: clearinghouse
    database: RAW_PRD
    schema: CLEARINGHOUSE
    description: |
      Historical PeMS data loaded from the clearinghouse feeds at
      https://pems.dot.ca.gov/feeds/clhouse/
    tables:
      - name: STATION_META
        description: Metadata for vehicle detector stations.
        columns:
          - name: FILENAME
          - name: SAMPLE_TIMESTAMP
          - name: SAMPLE_DATE
          - name: ID
          - name: FLOW_1
          - name: etc...
3

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.