The problem I’m having
I’m using dbt Unit tests on models built in Databricks. When I expect a column of type array, the unit tests produce SQL-statements with syntax errors.
Some example code or error messages
Model to test
select array("1", "2", "3") as my_array
Unit test
unit_tests:
- name: test_array
model: test_array_column_unit_test
given:
- input: ref('source')
rows: []
expect:
rows:
- my_array:
- 1
- 2
- 3
Error message:
Runtime Error in unit_test test_array (models/unit_tests.yml)
An error occurred during execution of unit test 'test_array'. There may be an error in the unit test definition: check the data types.
Database Error
[PARSE_SYNTAX_ERROR] Syntax error at or near '['. SQLSTATE: 42601 (line 15, pos 16)
== SQL ==
/* {"app": "dbt", "dbt_version": "1.8.7", "dbt_databricks_version": "1.8.7", "databricks_sql_connector_version": "3.1.2", "profile_name": "sdp_transform", "target_name": "dev", "node_id": "unit_test.sdp_transform.test_array_column_unit_test.test_array"} */
-- Build actual result given inputs
with dbt_internal_unit_test_actual as (
select
my_array, 'actual' as `actual_or_expected`
from (
select array("1", "2", "3") as my_array
) _dbt_internal_unit_test_actual
),
-- Build expected result
dbt_internal_unit_test_expected as (
select
my_array, 'expected' as `actual_or_expected`
from (
select cast([1, 2, 3] as array<string>)
----------------^^^
as my_array
) _dbt_internal_unit_test_expected
)
-- Union actual and expected results
select * from dbt_internal_unit_test_actual
union all
select * from dbt_internal_unit_test_expected
06:48:55
06:48:55 Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1