I use dbt in Databricks. I have several spark sql queries using quoted identifiers with special characters. Example column name: change:DateCreate.
I encounter a parsing error when referring to this column in the unit test when I include the backticks.
I encounter a syntax error on the special character ‘:’ when I put the name in double quotes, i.e. “change:DateCreate”.
Yes, I know: best practice is not to use special characters in a column name. But hey, the SQL is valid and now the Unit Test cannot handle it. As a work-around: I will change the column name not to include a special character. I hope my comments trigger a solution, though.