Hey guys, hope you’re all doing well!
I’m trying to update my dbt-core version from 0.19.0 to 1.7.2. I have a running project with 700+ models on it. To do this, i’m defining a new docker image with necessary packages and newer versions. This image will be uploaded to AWS and everything will run on ECS.
While trying to update the docker image locally first, i’m getting some errors when trying to run any command.
I managed to successfully update dbt and python and all deps, and my dbt debug shows no errors. this is the output:
15:43:16 Running with dbt=1.7.2
15:43:16 dbt version: 1.7.2
15:43:16 python version: 3.9.18
15:43:16 python path: /usr/local/bin/python
15:43:16 os info: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.36
15:43:16 Using profiles dir at /sg-dbt/config
15:43:16 Using profiles.yml file at /sg-dbt/config/profiles.yml
15:43:16 Using dbt_project.yml file at /sg-dbt/dbt_project.yml
15:43:16 adapter type: bigquery
15:43:16 adapter version: 1.7.2
15:43:16 Configuration:
15:43:16 profiles.yml file [OK found and valid]
15:43:16 dbt_project.yml file [OK found and valid]
15:43:16 Required dependencies:
15:43:16 - git [OK found]
15:43:16 Connection:
15:43:16 method: service-account
15:43:16 database: database
15:43:16 execution_project: projext
15:43:17 schema: fernandop_dbt
15:43:17 location: None
15:43:17 priority: interactive
15:43:17 maximum_bytes_billed: None
15:43:17 impersonate_service_account: None
15:43:17 job_retry_deadline_seconds: None
15:43:17 job_retries: 1
15:43:17 job_creation_timeout_seconds: None
15:43:17 job_execution_timeout_seconds: 300
15:43:17 keyfile: config/bq_service_account.json
15:43:17 timeout_seconds: 300
15:43:17 refresh_token: None
15:43:17 client_id: None
15:43:17 token_uri: None
15:43:17 dataproc_region: None
15:43:17 dataproc_cluster_name: None
15:43:17 gcs_bucket: None
15:43:17 dataproc_batch: None
15:43:17 Registered adapter: bigquery=1.7.2
15:43:18 Connection test: [OK connection ok]
those are my packages:
packages:
- package: “dbt-labs/dbt_utils”
version: 1.1.1 - package: elementary-data/elementary
version: 0.13.2
To test, i’m trying to run a dummy model with a specific macro as I think macro sytax is what’s breaking the runs. I have a macro which uses adapter_dispatch defined like this:
{%- macro access_array_element(col_name, index) -%}
{{adapter.dispatch(‘access_array_element’)(col_name, index)}}
{%- endmacro -%}
{%- macro default__access_array_element(col_name, index) -%}
{{col_name}}[{{index}}]
{%- endmacro -%}
{%- macro bigquery__access_array_element(col_name, index) -%}
{{col_name}}[ORDINAL({{index}})]
{%- endmacro -%}
and this is my dummy model, named test_access_array_element.sql
– test_access_array_element.sql
with test_data as (
select ‘[“apple”, “banana”, “cherry”]’::json as fruit_array
)
select
{{ access_array_element(‘fruit_array’, 2) }} as second_fruit
from test_data
this is the output i get from running dbt run --select test_access_array_element
root@a67160438635:/sg-dbt# dbt run --select test_access_array_element
15:27:59 Running with dbt=1.7.2
15:28:00 Registered adapter: bigquery=1.7.2
15:28:00 Unable to do partial parsing because saved manifest not found. Starting full parse.
15:28:03 Encountered an error:
‘Name’ object has no attribute ‘value’
15:28:03 Traceback (most recent call last):
File “/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py”, line 90, in wrapper
result, success = func(*args, **kwargs)
File “/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py”, line 75, in wrapper
return func(*args, **kwargs)
File “/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py”, line 168, in wrapper
return func(*args, **kwargs)
File “/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py”, line 197, in wrapper
return func(*args, **kwargs)
File “/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py”, line 244, in wrapper
return func(*args, **kwargs)
File “/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py”, line 271, in wrapper
manifest = ManifestLoader.get_full_manifest(
File “/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py”, line 318, in get_full_manifest
manifest = loader.load()
File “/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py”, line 443, in load
self.load_and_parse_macros(project_parser_files)
File “/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py”, line 634, in load_and_parse_macros
self.macro_depends_on()
File “/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py”, line 728, in macro_depends_on
possible_macro_calls = statically_extract_macro_calls(
File “/usr/local/lib/python3.9/site-packages/dbt/clients/jinja_static.py”, line 43, in statically_extract_macro_calls
ad_macro_calls = statically_parse_adapter_dispatch(
File “/usr/local/lib/python3.9/site-packages/dbt/clients/jinja_static.py”, line 97, in statically_parse_adapter_dispatch
func_name = func_call.args[0].value
AttributeError: ‘Name’ object has no attribute ‘value’
does anyone knows if jinja syntax is drastically different from one ver to the other? the project has LOTS of macros defined and creating it from scratch is not an option as the team is way smaller than it used to be. also all developers involved in the creation of this 0.19 dbt env aren’t on the company anymore.
Thanks for any answers!