Update existing table descriptions in BQ from dbt

Help
1

I have enabled persist_docs in dbt_project.yml and provided table/column descriptions in schema.yml files. It’s able to update column descriptions but not working for already existing tables. It puts table descriptions when only table is created. How can I update table descriptions for already existing tables?

Thanks!

dbt_project.yml

models:
  +persist_docs:
    relation: true
    columns: true
2

This is correct, that’s just how it works. Hopefully your database has an API you can use?

Note: @Mike Stanley originally posted this reply in Slack. It might not have transferred perfectly.

3

Thanks for the response. Yes there are other options I will check.

Note: @mustafasaltikk originally posted this reply in Slack. It might not have transferred perfectly.

4

thanks for posting this, i asked the exact same question a little over 12 hours ago and didn’t get a response.

in my case, i tried doing ALTER TABLE my_table ALTER COLUMN my_column SET OPTIONS(description=“my_description”); directly in the GCP console and it worked on my existing incremental tables. so i think i will just write a little python script in colab to loop through all my .yml files and run that SQL.

i’m genuinely curious why dbt doesn’t just run the same SQL for persist_docs. if there are at least two of us who expected that behavior then i’m sure we’re not the only ones.

Note: @LKG originally posted this reply in Slack. It might not have transferred perfectly.

5

The adapter could probably be modified to do this, but the simple answer is that it doesn’t amend the table unless it’s full-refreshing. I think it’s related to the way on_schema_change is handled.

If I were going to change it I’d go take a look at the adapter’s code and figure out when and how it does the update - it can presumably be changed to run on every dbt run

Note: @Mike Stanley originally posted this reply in Slack. It might not have transferred perfectly.

6

oh, interesting! i have on_schema_change='fail', i had tried the full refresh to see if it would update, but it only updated the table-level description and not the columns. i’ll try temporarily removing the on_schema_change config and see if that updates the columns with the full refresh.

i did poke around github for a while last night and found this, but haven’t dug any deeper yet: https://github.com/dbt-labs/dbt-bigquery/blob/6c0afe4cfb69761dada5d16150fe632b8f72bf39/dbt/adapters/bigquery/impl.py#L609|https://github.com/dbt-labs/dbt-bigquery/blob/6c0afe4cfb69761dada5d16150fe632b8f72bf39/dbt/adapters/bigquery/impl.py#L609

Note: @LKG originally posted this reply in Slack. It might not have transferred perfectly.