I just set up the dbt-databricks adapter and dbt is failing to create a model (TABLE) in an existing Databricks SCHEMA.
Runtime Error
[INSUFFICIENT_PERMISSIONS] Insufficient privileges:
User does not have permission CREATE on CATALOG.
In my configuration.yml, I’ve specified the SCHEMA name to be used for the model and this SCHEMA already exists in Databricks.
I have the following permissions:
CATALOG => USE CATALOG (but not CREATE SCHEMA)
SCHEMA => ALL PRIVILEGES (including CREATE/MODIFY/DELETE TABLES)
which I thought should be sufficient since the SCHEMA already exists - but the dbt error indicates it is somehow trying to CREATE some SCHEMA in the CATALOG.
Our Admin temporarily granted me CREATE privilege on CATALOG - after which dbt was able to create the model (TABLE) in the existing SCHEMA.
After removing the CREATE privilege on CATALOG, same error recurs.
Please can someone explain why dbt requires CREATE on CATALOG privilege in order to create a TABLE in a SCHEMA in the CATALOG.
Same issue: database + schema exists, but it tries to create it anyway and let me bump into permission issues. Any workaround?