[v0.17 branch] dbt compile does more than just compiling

Archive
1

Hi,

I am currently trying to write a PR for dbt, so I was using dbt on the dev/marian-anderson branch which looks like the most up-to-date branch.

While testing the dbt compile command, I got hit by an error message that says

User does not have bigquery.datasets.create permission in project

And from what I saw in the code it indeed looks like the dbt compile calls a method create_schemas() that tries to create datasets on the BigQuery project.

I question whether this is a good thing to do that, especially when the description of the compile command simply says:

Generates executable SQL from source, model, test, and analysis files. Compiled SQL files are written to the target/ directory.

We are using an older version of dbt in production and using the dbt compile command in PR-triggered-builds to validate the syntax without having to connect to BigQuery, but I suspect that it might cease working when we upgrade dbt.

I just checked on v0.17 and it looks like it does the same thing there too.
If this is the intended behavious maybe you should update the documentation to explain
that the compile command does more than just compiling.

2

Creating schemas when running dbt compile is indeed not the intended behavior. This was a regression in 0.17.0 that we’re fixing in 0.17.1, to be released very soon. Check out the issue (#2525) and PR (#2528).

3

Thank you very much for your answer!

4

@jerco I might have missed something, but I pulled the latest version of the dev/marian-anderson branch and I am still getting the same access error while running dbt compile, yet the PR you mention seems to have already been merged into this branch already… maybe the bug was reintroduced somehow ?

EDIT: my bad, this is not the same error. I am getting an access denied for the bigquery.tables.list permission. Looks like it happens in
before_run > populate_adapter_cache > list_relations_without_caching

It might be normal … Is the dbt compile command supposed to require read-only access on the target database ?

5

Correct, dbt compile does require a database connection, and sufficient permissions to read metadata about objects in the database. It should not require permissions to create objects in the database, nor should it attempt to do so (i.e. the regression you mention in the original post).