Use in dbt the bigquery dataset option storage_billing_model to PHYSICAL

Help
,
1

I can’t find the Bigquery dataset option storage_billing_model to use PHYSICAL instead of LOGICAL (default) storage of Bigquery when using dbt core (version 0.25)

The physical option will be able to lower our cost of storage

I’ve already look into the documentation and didn’t find all i needed, I found this to configure the profiles.yaml file:

my-bigquery-db:
target: dev
outputs:
dev:
type: bigquery
method: oauth
project: GCP_PROJECT_ID
dataset: DBT_DATASET_NAME # You can also use "schema" here
threads: 4 # Must be a value of 1 or greater
[OPTIONAL_CONFIG](https://docs.getdbt.com/docs/core/connect-data-platform/bigquery-setup#optional-configurations): VALUE

but there is no specification of OPTIONAL_CONFIG of the option storage_billing_model of Bigquery

Thanks a lot for the help if someone knows how to switch to using the PHYSICAL storage using dbt core

2

Billing model is a dataset-level setting. dbt doesn’t have a first-class entity for datasets so there’s nowhere for it to hang that config. This, and other dataset-level configs, are a common request but it’s simply not possible today. I would recommend managing datasets outside dbt and using that tool (eg Terraform) to handle billing setup

Note: @Mike Stanley originally posted this reply in Slack. It might not have transferred perfectly.

3

Hi, but when using profiles.yaml , if you configure the dataset there to connect when using dbt (—target) and that the dataset doesn’t exist, will it create the dataset ?

4

It will, but nevertheless there is no way to attach configs to this because dbt doesn’t have a first-class concept of datasets.

Note: @Mike Stanley originally posted this reply in Slack. It might not have transferred perfectly.

5

Ok thank you for the clarification, have a good day

6

I would think you’d want to monitor this decision on a case-by-case basis? In that example, maybe use a post hook macro with an alter schema to change the storage billing model config on a single table/model

Note: @Tony Tushar originally posted this reply in Slack. It might not have transferred perfectly.