TypeError: can not serialize 'Undefined' object

Help
1

The problem

I work on some dbt models locally using-dbt core. Until yesterday everything worked fine. At some point of time commands like dbt build, dbt test etc. started to return error as displayed below. I have no idea what can I do to fix that problem.

Tech context:
OS: Ubuntu24.04 (WSL)
Python: 3.11.0
dbt-core: 1.7.10
databricks:1.7.10
virtual environment: poetry 1.8.1

I took following steps and all of them didn’t help:

  • Recreated virtual environment
  • Reinstalled python
  • Reinstalled msgpack
  • Run commands with --no-partial-parse

Error message

07:25:47  Encountered an error:
can not serialize 'Undefined' object
07:25:47  Traceback (most recent call last):
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/cli/requires.py", line 91, in wrapper
    result, success = func(*args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/cli/requires.py", line 76, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/cli/requires.py", line 169, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/cli/requires.py", line 198, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/cli/requires.py", line 245, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/cli/requires.py", line 271, in wrapper
    ctx.obj["manifest"] = parse_manifest(
                          ^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/parser/manifest.py", line 1798, in parse_manifest
    manifest = ManifestLoader.get_full_manifest(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/parser/manifest.py", line 318, in get_full_manifest
    manifest = loader.load()
               ^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/parser/manifest.py", line 567, in load
    self.write_manifest_for_partial_parse()
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/parser/manifest.py", line 752, in write_manifest_for_partial_parse
    manifest_msgpack = self.manifest.to_msgpack(extended_mashumaro_encoder)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 71, in __mashumaro_to_msgpack__
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/dbt/parser/manifest.py", line 139, in extended_mashumaro_encoder
    return msgpack.packb(data, default=extended_msgpack_encoder, use_bin_type=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mateusz/databricks/.venv/lib/python3.11/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 294, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 300, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 297, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  [Previous line repeated 1 more time]
  File "msgpack/_packer.pyx", line 291, in msgpack._cmsgpack.Packer._pack
TypeError: can not serialize 'Undefined' object
2

This can be a lot of stuff, but the most common problem I see with this error is not using quotes inside the config block

So, for example, this

{{ config(
    materialized=incremental
) }}

Should be

{{ config(
    materialized='incremental'
) }}

Or this

{{ config(
    options={
        partition_by : ['PropertySetDetailSeasonID']
    }   
) }}

Should be

{{ config(
    options={
       'partition_by': ['PropertySetDetailSeasonID']
    }   
) }}

Check if you have something like that

2 Likes
3

That was the reason for the error. Thank you!

4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.