Package installation fails on Windows when directory paths are very long

[bakerbryce]

Posting for awareness in case anyone else runs into a similar issue.

Problem
Environment summary:

• Azure DevOps Self-Hosted Agent
• Windows Server 2019
• Python 3.9.10 and 3.10
• dbt_utils v0.9.2
• Running locally with no Python virtual environment (works fine in venv and local developer machines)
• Running dbt deps command using PowerShell 2.0

Error below occurs on agent when attempting to install dbt packages.

09:08:52  Running with dbt=1.2.1
09:08:52  Installing dbt-labs/dbt_utils
09:08:53  Encountered an error:
[Errno 2] No such file or directory: 'dbt_packages\\dbt-utils-0.9.2\\integration_tests\\data\\schema_tests\\data_test_mutually_exclusive_ranges_with_gaps_zero_length.csv'
09:08:53  Traceback (most recent call last):
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\main.py", line 129, in main
    results, succeeded = handle_and_check(args)
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\main.py", line 191, in handle_and_check
    task, res = run_from_args(parsed)
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\main.py", line 238, in run_from_args
    results = task.run()
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\task\deps.py", line 67, in run
    package.install(self.config, renderer)
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\deps\registry.py", line 74, in install
    connection_exception_retry(download_untar_fn, 5)
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\utils.py", line 616, in _connection_exception_retry
    return fn()
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\deps\registry.py", line 86, in download_and_untar
    system.untar_package(tar_path, deps_path, package_name)
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\site-packages\dbt\clients\system.py", line 478, in untar_package
    tarball.extractall(dest_dir)
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\tarfile.py", line 2040, in extractall
    self.extract(tarinfo, path, set_attrs=not tarinfo.isdir(),
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\tarfile.py", line 2081, in extract
    self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\tarfile.py", line 2154, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "D:\AzureDevOps\_work\_tool\Python\3.10.0\x64\lib\tarfile.py", line 2195, in makefile
    with bltn_open(targetpath, "wb") as target:
FileNotFoundError: [Errno 2] No such file or directory: 'dbt_packages\\dbt-utils-0.9.2\\integration_tests\\data\\schema_tests\\data_test_mutually_exclusive_ranges_with_gaps_zero_length.csv'

[bakerbryce]

Solution

The error may be caused by Windows not allowing long file paths by default. You can resolve this on the agent with the below PowerShell command.
Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem -Name LongPathsEnabled -Value 1;