The problem I’m having
Hi everyone,
I’m relatively new to the dbt world and I’m currently maintaining a dbt-adapter. Recently there was a bug which prevented adding quotes to an identifier when quoting was enabled.
After fixing it, I added a test for it. My test itself is a modified version of TestIncremental from dbt-core. All I did was to change the model name “incremental.sql” to a reserved keyword in my database and in the test_increment method I changed all the references of incremental to the reserved keyword I’m using.
Now when I run the test, during one of the dbt runs the test fails with the following error
Invoking dbt with ['seed']
12:26:16 Running with dbt=1.8.9
12:26:16 Registered adapter: impala=1.8.0
12:26:16 Unable to do partial parsing because saved manifest not found. Starting full parse.
12:26:18 Found 1 model, 2 seeds, 1 source, 441 macros
12:26:18
12:26:45 Concurrency: 4 threads (target='default')
12:26:45
12:26:45 1 of 2 START seed file test17387583538034804709_test_basic.added ............... [RUN]
12:26:45 2 of 2 START seed file test17387583538034804709_test_basic.base ................ [RUN]
12:26:58 2 of 2 OK loaded seed file test17387583538034804709_test_basic.base ............ [INSERT 10 in 13.06s]
12:26:59 1 of 2 OK loaded seed file test17387583538034804709_test_basic.added ........... [INSERT 20 in 13.07s]
12:27:02
12:27:02 Finished running 2 seeds in 0 hours 0 minutes and 43.62 seconds (43.62s).
12:27:02
12:27:02 Completed successfully
12:27:02
12:27:02 Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
Invoking dbt with ['run', '--vars', 'seed_name: base']
12:27:24 Running with dbt=1.8.9
12:27:24 Registered adapter: impala=1.8.0
12:27:24 Unable to do partial parsing because config vars, config profile, or config target have changed
12:27:25 Found 1 model, 2 seeds, 1 source, 441 macros
12:27:25
12:28:33 Concurrency: 4 threads (target='default')
12:28:33
12:28:33 1 of 1 START sql incremental model test17387583538034804709_test_basic.select .. [RUN]
12:28:44 1 of 1 OK created sql incremental model test17387583538034804709_test_basic.select [OK in 10.48s]
12:28:47
12:28:47 Finished running 1 incremental model in 0 hours 1 minutes and 21.97 seconds (81.97s).
12:28:47
12:28:47 Completed successfully
12:28:47
12:28:47 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
Invoking dbt with ['run', '--vars', 'seed_name: added']
12:29:13 Running with dbt=1.8.9
12:29:13 Registered adapter: impala=1.8.0
12:29:14 Unable to do partial parsing because config vars, config profile, or config target have changed
12:29:14 Found 1 model, 2 seeds, 1 source, 441 macros
12:29:14
12:30:49 Impala adapter: Unable to fetch relation type test17387583538034804709_test_basic.select: Runtime Error
Unable to establish connection to Impala server: Query 7646b12f95a54a62:da855e4c00000000 failed:
ParseException: Syntax error in line 3:
...83538034804709_test_basic.select
^
Encountered: SELECT
Expected: IDENTIFIER
Hint: reserved words have to be escaped when used as an identifier, e.g. `select`
CAUSED BY: Exception: Syntax error
What makes this problem interesting to me is that during first dbt-run that passes, the describe table command gets a relation name that is already quoted
describe extended test17387583538034804709_test_basic.`select`
But in the second dbt run the describe command wasn’t quoted. So I spent some time debugging and I couldn’t quite figure out where the relation/table name gets this quote. To me it looks like this is happening from the dbt library and the adapter implementation just defines what the quote character is.
This is only happening with the describe command and the identifier is always quoted for “create table” statements.
Can someone tell me what I’m missing here? Any helpful pointers would be appreciated.