On BigQuery, dbt can perform (authorize) ‘Authorized Views’. Since BigQuery also offers the more scaled and code-efficient ‘Authorized Datasets’, I recommend a dbt enhancement so dbt can authorize ‘Authorized Datasets’. The simplest way to begin, I believe, is simply to setup Auth’d Datasets just like Auth’d views. A more scaled approach, though, would be to allow for specification of (a) one authorized (downstream) dataset to be authorized, (b) multiple BigQuery projects (prod, uat, dev environments, in our case), and then for the ‘authorizing’ dataset, allow for string-matching ‘authorizing’ (upstream) dataset names (perhaps for example ‘ERP_Sys%’ would grant authorization for the one downstream dataset in the following upstream datasets: ERP_Sys_Schema_A, ERP_Sys_Schema B, …C, D, etc). Thoughts?
Related topics
Topic | Replies | Views | Activity | |
---|---|---|---|---|
Managing Authorization in Google BigQuery | 1 | 3635 | May 14, 2021 | |
Set grants on dataset level in BigQuery | 4 | 1691 | October 4, 2024 | |
Grant & revoke in Big query | 3 | 1523 | January 11, 2023 | |
Can we use multiple Bigquery datasets (or projects) in the same dbt project? | 4 | 15798 | November 23, 2020 | |
Test DBT Cloud (free version) over a personnal Bigquery Personnal project | 0 | 623 | December 7, 2023 |