KeyError in dbt_snowflake within docker container

Help
, ,
1

The problem I’m having

Trying to run dbt using a docker image, which keeps throwing a “dbt_snowflake://macros/adapters.sql” error

dbt=1.9.1
Registered adapter: snowflake=1.9.0

This is a very simple project with one model, which works locally (vscode).

The context of why I’m trying to do this

Attempting to get a working docker image and configuration, on top of which I can build a project

What I’ve already tried

I’ve tried multiple python version (3.9 - 3.13, slim and regular)

I’ve tried a few different approaches to the dockerfile. Below are some examples.

Some example code or error messages

Full error below:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 153, in wrapper
    result, success = func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 235, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 264, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 327, in wrapper
    setup_manifest(ctx, write=write, write_perf_info=write_perf_info)
  File "/usr/local/lib/python3.9/site-packages/dbt/cli/requires.py", line 351, in setup_manifest
    ctx.obj["manifest"] = parse_manifest(
  File "/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py", line 2061, in parse_manifest
    manifest = ManifestLoader.get_full_manifest(
  File "/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py", line 312, in get_full_manifest
    manifest = loader.load()
  File "/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py", line 381, in load
    self.load_and_parse_macros(project_parser_files)
  File "/usr/local/lib/python3.9/site-packages/dbt/parser/manifest.py", line 682, in load_and_parse_macros
    block = FileBlock(self.manifest.files[file_id])
KeyError: 'dbt_snowflake://macros/adapters.sql'

Dockerfile:

# Use a base image with Python and necessary dependencies
FROM python:3.9-slim

# Install system dependencies (if needed)
RUN apt-get update -y && apt-get install -y --no-install-recommends build-essential \
    git && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Set the working directory inside the container
WORKDIR /usr/app

# Copy your dbt project files into the container
COPY . .

# Install dbt and any required adapters (e.g., for Snowflake, BigQuery, etc.)
RUN pip install --upgrade pip
RUN pip install dbt-core dbt-snowflake -U

# Set the entrypoint to dbt
ENTRYPOINT ["dbt"]
2

This is resolved. The issue was with the Partial Parsing behavior. Partial Parsing is on by default. The install of dbt core and snowflake introduces a … ‘corrupted’ (can’t think of a better word) info in the /target files (especially partial_parse.msgpack).

The solution for this was to add these commands to the dockerfile AFTER the pip install of dbt and the snowflake adapter

RUN dbt deps
RUN dbt clean --profiles-dir /usr/app
RUN dbt compile --profiles-dir /usr/app

The clean will remove all files from the /target directory
The compile will recompile and recreate the manifest based on the installed files, which include the core, adapter and project files.