Add library when running DBT Python model on Databricks cluster

Help
1

The problem I’m having

I’m trying to run dbt python models on a Databricks job cluster. I want to add a dependent library to the job cluster, but it seems a dependent library cannot be specified in the “new_cluster” config (Databricks REST API reference). Is it possible to run dbt models on a Databricks job cluster and add libraries to the cluster?

What I’ve already tried

def model(dbt, session):
    dbt.config(
        submission_method="job_cluster",
        job_cluster_config={
            ...
            # where to add the library config?
        }
    )
2

Seems like one solution is to install dependent libraries in an init-script, and reference the init-script in the job_cluster_config as such:

def model(dbt, session):
    dbt.config(
        submission_method="job_cluster",
        job_cluster_config={
            ...
            "init_scripts": [
                {
                    "workspace": {
                        "destination": "my-init-script.sh"
                    }
                }
            ],
        }
    )

my-init-script.sh:

#!/bin/bash
pip install lib1
pip install lib2