The problem I’m having
I have a small dbt project where I am utilizing DuckDB (dev) and MotherDuck (prod).
Situation is as follows:
- Create DuckDB/ MotherDuck database and populate it
- Execute dbt build
- Try to execute dbt build again
Using DuckDB all three steps work just fine, but on MotherDuck step three fails.
The error happens once dbt build tries to build an incremental model, before things are working just fine.
The error I am getting is: Unhandled error while executing
What I’ve already tried
I tried to figure what the difference is between the execution in DuckDB/ MotherDuck.
But there should not be any difference at all:
DuckDB/ MotherDuck version: 1.1.0
dbt-core: 1.8.6
dbt-duckdb: 1.8.3
Python: 3.12.5
In both cases execution happens using a virtual environment.
Some example code or error messages
Its seems like the error is that there is no SQL code retrieved from the manifest?
This is the full error message:
Traceback (most recent call last):
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/task/base.py", line 370, in safe_run
result = self.compile_and_execute(manifest, ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/task/base.py", line 316, in compile_and_execute
result = self.run(ctx.node, manifest)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/task/base.py", line 419, in run
return self.execute(compiled_node, manifest)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/task/run.py", line 298, in execute
result = MacroGenerator(
^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/clients/jinja.py", line 84, in __call__
return self.call_macro(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt_common/clients/jinja.py", line 298, in call_macro
return macro(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 768, in __call__
return self._invoke(arguments, autoescape)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 782, in _invoke
rv = self._func(*arguments)
^^^^^^^^^^^^^^^^^^^^^^
File "<template>", line 137, in macro
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/sandbox.py", line 394, in call
return __context.call(__obj, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 303, in call
return __obj(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/clients/jinja.py", line 84, in __call__
return self.call_macro(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt_common/clients/jinja.py", line 298, in call_macro
return macro(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 768, in __call__
return self._invoke(arguments, autoescape)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 782, in _invoke
rv = self._func(*arguments)
^^^^^^^^^^^^^^^^^^^^^^
File "<template>", line 33, in macro
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/sandbox.py", line 394, in call
return __context.call(__obj, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 303, in call
return __obj(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/clients/jinja.py", line 84, in __call__
return self.call_macro(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt_common/clients/jinja.py", line 298, in call_macro
return macro(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 768, in __call__
return self._invoke(arguments, autoescape)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 782, in _invoke
rv = self._func(*arguments)
^^^^^^^^^^^^^^^^^^^^^^
File "<template>", line 52, in macro
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/sandbox.py", line 394, in call
return __context.call(__obj, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/jinja2/runtime.py", line 303, in call
return __obj(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/adapters/base/impl.py", line 356, in execute
return self.connections.execute(sql=sql, auto_begin=auto_begin, fetch=fetch, limit=limit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/adapters/duckdb/connections.py", line 119, in execute
return super().execute(sql, auto_begin, fetch, limit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/adapters/sql/connections.py", line 156, in execute
sql = self._add_query_comment(sql)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/adapters/base/connections.py", line 393, in _add_query_comment
return self.query_header.add(sql)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/adapters/base/query_headers.py", line 88, in add
return self.comment.add(sql)
^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/analytics/.venv/lib/python3.12/site-packages/dbt/adapters/base/query_headers.py", line 37, in add
if sql[-1] == ";":
Any idea whats going on here?