The problem I’m having
After upgrading dbt-core to v1.5 I’m getting parsing errors in models that previously had no issues. Nothing has changed in the repo since the upgrade. The issue seems to arise in models that use a 2 argument ref.
What I’ve already tried
Aftre reading a post in the dbt-Slack workspace I tried running dbt-clean, dbt-deps, dbt-debug and then running dbt-parse again but still the same issue. I tried re-running the same operation as just described but deleting the “target” directory (after dbt clean) but with the same results.
I checked the dbt.log file and saw that the parse operation always had an error after parsing one particular model. So I checked the jinja refs, model config and SQL code in this model and all the models in the same directory that were further down the alphabetical list. After removing a few comments and commented out jinja refs (i.e. {#{ ref('model') }#} I re-ran dbt parse but with the same result (error).
I’ve tried running dbt parse with dbt-core v1.4.0 and there are absolutely no issues, so I’m confident this issue is directly related to the new release.
Some important things to note:
- We use a multi-repo approach so we have 2 argument refs (e.g.
{{ ref('my_second_dbt_project', 'my_funky_model') }}) in lots of our models - I’ve tried running
dbt depsin several of our dbt projects (all in their own Git repos and separate) and I’m getting the same error across several of them- I think the issue might be specifically in the downstream projects (see above re: 2 argument refs): when I run
dbt parseanddbt compilein dbt projects that do not reference any other dbt projects it seems to run error-free
- I think the issue might be specifically in the downstream projects (see above re: 2 argument refs): when I run
- I made sure that all relevant package dependencies that
dbt-corehas were updated to the latest version
Some example code or error messages
Below is the CLI error message I keep getting
(dbt-env) PS C:\Users\user.name\Documents\GitLab Repositories\dbt-project-1> dbt parse
13:08:55 Running with dbt=1.5.0
13:08:56 Unable to do partial parsing because saved manifest not found. Starting full parse.
13:08:58 Encountered an error:
list index out of range
13:08:58 Traceback (most recent call last):
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 86, in wrapper
result, success = func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 71, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 142, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 168, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 215, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 242, in wrapper
manifest = ManifestLoader.get_full_manifest(
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\manifest.py", line 227, in get_full_manifest
manifest = loader.load()
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\manifest.py", line 386, in load
self.parse_project(
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\manifest.py", line 515, in parse_project
parser.parse_file(block)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\base.py", line 433, in parse_file
self.parse_node(file_block)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\base.py", line 394, in parse_node
self.render_update(node, config)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\models.py", line 337, in render_update
self.populate(node, config, statically_parsed)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\models.py", line 495, in populate
package, name = ref[0], refs[1]
IndexError: list index out of range
This is a copy of the logging information found in dbt.log:
============================== 15:30:02.996105 | 50de484e-22f3-4030-acf0-b19cc0f5ceb5 ==============================
e[0m15:30:02.996105 [info ] [MainThread]: Running with dbt=1.5.0
e[0m15:30:02.996105 [debug] [MainThread]: running dbt with arguments {'static_parser': 'True', 'target_path': 'None', 'cache_selected_only': 'False', 'indirect_selection': 'eager', 'send_anonymous_usage_stats': 'True', 'debug': 'False', 'warn_error': 'None', 'write_json': 'True', 'warn_error_options': 'WarnErrorOptions(include=[], exclude=[])', 'log_cache_events': 'False', 'printer_width': '80', 'use_experimental_parser': 'False', 'log_format': 'default', 'introspect': 'True', 'partial_parse': 'True', 'no_print': 'None', 'fail_fast': 'False', 'log_path': 'C:\\Users\\user.name\\Documents\\GitLab Repositories\\dbt-project-1\\logs', 'use_colors': 'True', 'version_check': 'True', 'quiet': 'False', 'profiles_dir': 'C:\\Users\\user.name\\Documents\\GitLab Repositories\\dbt-project-1'}
e[0m15:30:03.511455 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'project_id', 'label': '50de484e-22f3-4030-acf0-b19cc0f5ceb5', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000013A0C90D430>]}
e[0m15:30:03.527078 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'adapter_info', 'label': '50de484e-22f3-4030-acf0-b19cc0f5ceb5', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000013A0C945AC0>]}
e[0m15:30:03.608310 [debug] [MainThread]: checksum: 4568eb639a77b8fcb3a1f4a07856f42b1ff63f1376652889143968e1dbdafbda, vars: {}, profile: , target: , version: 1.5.0
e[0m15:30:03.608310 [info ] [MainThread]: Unable to do partial parsing because saved manifest not found. Starting full parse.
e[0m15:30:03.608310 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'partial_parser', 'label': '50de484e-22f3-4030-acf0-b19cc0f5ceb5', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000013A09C2B2E0>]}
e[0m15:30:06.768524 [debug] [MainThread]: 1699: static parser successfully parsed benchmarking\add_calculated_fields.sql
e[0m15:30:06.768524 [debug] [MainThread]: 1699: static parser successfully parsed benchmarking\calculate_rate_metrics.sql
e[0m15:30:06.783262 [debug] [MainThread]: 1699: static parser successfully parsed benchmarking\clean_in_geo_report.sql
e[0m15:30:06.784262 [debug] [MainThread]: 1699: static parser successfully parsed benchmarking\cm_data_join.sql
e[0m15:30:06.784262 [debug] [MainThread]: 1699: static parser successfully parsed benchmarking\pre_process_plan_line_info.sql
e[0m15:30:06.784262 [debug] [MainThread]: 1699: static parser successfully parsed brand_lift_study\bls_allchannels_union.sql
e[0m15:30:06.784262 [debug] [MainThread]: 1699: static parser successfully parsed brand_lift_study\bls_display_all_other_platforms.sql
e[0m15:30:06.784262 [debug] [MainThread]: 1699: static parser successfully parsed brand_lift_study\bls_display_dv360.sql
e[0m15:30:06.800110 [debug] [MainThread]: 1699: static parser successfully parsed brand_lift_study\bls_display_twitter.sql
e[0m15:30:06.800110 [debug] [MainThread]: 1699: static parser successfully parsed brand_lift_study\bls_dv360_agg.sql
e[0m15:30:06.800110 [error] [MainThread]: Encountered an error:
list index out of range
e[0m15:30:06.800110 [error] [MainThread]: Traceback (most recent call last):
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 86, in wrapper
result, success = func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 71, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 142, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 168, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 215, in wrapper
return func(*args, **kwargs)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\cli\requires.py", line 242, in wrapper
manifest = ManifestLoader.get_full_manifest(
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\manifest.py", line 227, in get_full_manifest
manifest = loader.load()
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\manifest.py", line 386, in load
self.parse_project(
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\manifest.py", line 515, in parse_project
parser.parse_file(block)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\base.py", line 433, in parse_file
self.parse_node(file_block)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\base.py", line 394, in parse_node
self.render_update(node, config)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\models.py", line 337, in render_update
self.populate(node, config, statically_parsed)
File "C:\Users\user.name\Anaconda3\envs\dbt-env\lib\site-packages\dbt\parser\models.py", line 495, in populate
package, name = ref[0], refs[1]
IndexError: list index out of range
e[0m15:30:06.815744 [debug] [MainThread]: Command `dbt parse` failed at 15:30:06.815744 after 3.96 seconds
e[0m15:30:06.815744 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000013A097520A0>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000013A0CD11640>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x0000013A0CD142B0>]}
e[0m15:30:06.815744 [debug] [MainThread]: Flushing usage events