Error setting up dbt-databricks environment (lz4 package installation failed)

Help
1

Hi Community Members, I’m new to dbt, trying to setup a environment on my local machine. Getting error at the step#3 when I try to execute the pip command

  1. Installed Python version 3.12.0, VS Code
  2. Created virtual python environment
  3. Ran command pip install dbt-databricks, getting below error :
    " × Building wheel for lz4 (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [15 lines of output]
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-cpython-312
    creating build\lib.win-amd64-cpython-312\lz4
    copying lz4\version.py → build\lib.win-amd64-cpython-312\lz4
    copying lz4_init_.py → build\lib.win-amd64-cpython-312\lz4
    creating build\lib.win-amd64-cpython-312\lz4\block
    copying lz4\block_init_.py → build\lib.win-amd64-cpython-312\lz4\block
    creating build\lib.win-amd64-cpython-312\lz4\frame
    copying lz4\frame_init_.py → build\lib.win-amd64-cpython-312\lz4\frame
    running build_ext
    building ‘lz4._version’ extension
    error: Microsoft Visual C++ 14.0 or greater is required. Get it with “Microsoft C++ Build Tools”: Microsoft C++ Build Tools - Visual Studio
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for lz4
Failed to build lz4
ERROR: Could not build wheels for lz4, which is required to install pyproject.toml-based projects"
A bit of Context why I’m trying to do this: Working on a POC project where I would like to use Azure Data Lake Storage, dbt and databricks to build a lakehouse architecture based data platform.

What I’ve already tried

  1. Checked the latest python version is installed.
  2. Installed Visual Studio Buil d tools 2022 (refer screenshot that shows the C++ versions installed on machine.
  3. tried restarting the machine after installation
    `
    Screenshots:

image
image1178×772 141 KB

2

Hi,
I don’t see the screenshot with visual studio version installed on the machine, but I had this error and following the instructions on this link Configure Visual Studio Code for Microsoft C++ fixed that. I hope this helps.

4

Thank you for getting back to me. I truly value the time you dedicated to responding. I wanted to update you that I managed to resolve the issue by uninstalling the latest Python version (3.12.0) and reinstalling the earlier version, Python 3.11.6. This seemed to fix the problem. It appears there might be an issue with installing the “dbt-databricks” adapter with Python 3.12.0. I’m uncertain if there are any known issues in this regard.

1 Like