The problem I’m having
Our project uses dbt version 1.2. When we execute dbt run -m ./models, it will only execute all the models under that folder. For example, there are 91 models in the folder “./models”, dbt run will only run these 91 models.
BUT, after upgrading to dbt 1.7, when I executed the same command, we found that three additional models were included:
-
logging.stg_dbt_audit_log
-
logging.default.stg_dbt_deployments
-
logging.default.stg_dbt_model_deployments
Then an error message appears:
08:49:45 Completed with 26 errors and 0 warnings:
08:49:45
08:49:45 Database Error in model stg_dbt_audit_log (models/stg_dbt_audit_log.sql)
001003 (42000): SQL compilation error:
syntax error line 22 at position 51 unexpected 'as'.
compiled Code at target/run/logging/models/stg_dbt_audit_log.sql
What I’ve already tried
I switched to a virtual environment with dbt 1.2, and indeed it runs only 91 models.
Upon further investigation, I found that these 3 models came from the package dbt-labs/logging referenced in packages.yml.
And also found that in the new version of dbt, there have been significant changes to packages.yml.
Refer to dbt doc here: Packages | dbt Developer Hub.
Some example code or error messages
My project structure is
My dbt_project.yml:
name: "xxxxx"
version: "1.0.0"
config-version: 2
require-dbt-version: [">=0.20.0"]
profile: "xxxxx"
model-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
seed-paths: ["data"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
target-path: "target"
clean-targets:
- "target"
- "dbt_packages"
- "./setup/target"
models:
...
seeds:
...
packages.yml is as follows:
packages:
- package: dbt-labs/logging
version: 0.6.0
- git: "https://github.com/Datavault-UK/dbtvault"
revision: v0.7.9
MY QUESTION
Considering that we have many referenced external packages, each containing some sample models, and we don’t want to execute those sample models but only leverage their functionality to execute our models, how can I skip these models?
Currently I use the --exclude command to skip them. Is there a better way to skip them?