# \#bigquery

**URL:** https://discourse.getdbt.com/tag/bigquery/36.md

[Latest](https://discourse.getdbt.com/latest.md) · [Categories](https://discourse.getdbt.com/categories.md) · [Tags](https://discourse.getdbt.com/tags.md)

---

## [metricflow self deployment](https://discourse.getdbt.com/t/metricflow-self-deployment/12671)

<div class="topic-metadata">

**Author:** [@adam.nowatkowski](https://discourse.getdbt.com/u/adam.nowatkowski)\
**Replies:** 2\
**Last updated:** [November 4, 2026, 7:56pm UTC](https://discourse.getdbt.com/t/metricflow-self-deployment/12671 "2026-11-04T19:56:34Z")

</div>

SelfDeployment option To this moment i have not found proper instruction for self deployment of semantic model from dbt core - dbt CLI. Due to fact that our main flow is in AirFlow on GCP. We started using dbt CLI with …

---

## [Built a SQL data debugging toolkit - looking for analysts to test it](https://discourse.getdbt.com/t/built-a-sql-data-debugging-toolkit-looking-for-analysts-to-test-it/20635)

<div class="topic-metadata">

**Author:** [@burzykowskianalytics](https://discourse.getdbt.com/u/burzykowskianalytics)\
**Replies:** 0\
**Last updated:** [March 18, 2026, 1:49pm UTC](https://discourse.getdbt.com/t/built-a-sql-data-debugging-toolkit-looking-for-analysts-to-test-it/20635 "2026-03-18T13:49:27Z")

</div>

I’ve been working on a structured SQL debugging workflow for analysts and BI developers. It consists of 30 SQL checks organized into a 7-step workflow (schema validation → nulls → duplicates → joins → distribution → fres…

---

## [BigQuery JS UDF (nanoid) returns same value many times only when run via dbt incremental MERGE](https://discourse.getdbt.com/t/bigquery-js-udf-nanoid-returns-same-value-many-times-only-when-run-via-dbt-incremental-merge/20566)

<div class="topic-metadata">

**Author:** [@jirick](https://discourse.getdbt.com/u/jirick)\
**Replies:** 0\
**Last updated:** [January 29, 2026, 7:23pm UTC](https://discourse.getdbt.com/t/bigquery-js-udf-nanoid-returns-same-value-many-times-only-when-run-via-dbt-incremental-merge/20566 "2026-01-29T19:23:20Z")

</div>

The problem I’m having I’m using a BigQuery JavaScript UDF (NanoID) to generate unique IDs in dbt models. For ~2 years it worked fine, but recently (starting around Oct 2024) some dbt runs produce the same generated ID r…

---

## [Here's a macro to keep your warehouse clean by automatically removing redundant or orphaned objects](https://discourse.getdbt.com/t/heres-a-macro-to-keep-your-warehouse-clean-by-automatically-removing-redundant-or-orphaned-objects/20562)

<div class="topic-metadata">

**Author:** [@albertovpd](https://discourse.getdbt.com/u/albertovpd)\
**Replies:** 0\
**Last updated:** [January 26, 2026, 1:36pm UTC](https://discourse.getdbt.com/t/heres-a-macro-to-keep-your-warehouse-clean-by-automatically-removing-redundant-or-orphaned-objects/20562 "2026-01-26T13:36:02Z")

</div>

For BigQuery users working with dbt This might be useful: Some time ago I developed a BigQuery autocleaner macro for dbt. It inspects your BigQuery datasets and removes any tables or views not declared as models in your…

---

## [dbt python model (using prophet library) on DataProc Serverless cannot connect to pypi](https://discourse.getdbt.com/t/dbt-python-model-using-prophet-library-on-dataproc-serverless-cannot-connect-to-pypi/20393)

<div class="topic-metadata">

**Author:** [@mgloel81](https://discourse.getdbt.com/u/mgloel81)\
**Replies:** 1\
**Last updated:** [November 11, 2025, 9:25am UTC](https://discourse.getdbt.com/t/dbt-python-model-using-prophet-library-on-dataproc-serverless-cannot-connect-to-pypi/20393 "2025-11-11T09:25:19Z")

</div>

The problem I’m having I am trying to run a python model that makes use of facebook’s prophet library to run time series prediction on one of my tables and write the predictions into a new table. The script itself is st…

---

## [add a time filter in DBT API run request](https://discourse.getdbt.com/t/add-a-time-filter-in-dbt-api-run-request/20320)

<div class="topic-metadata">

**Author:** [@DavidAichelin](https://discourse.getdbt.com/u/DavidAichelin)\
**Replies:** 1\
**Last updated:** [November 7, 2025, 3:58am UTC](https://discourse.getdbt.com/t/add-a-time-filter-in-dbt-api-run-request/20320 "2025-11-07T03:58:22Z")

</div>

Hello, I’d like to map the steps of my dbt jobs with their usage in BigQuery. To do this, I’m calling the dbt API using Airbyte. I’m currently using the following endpoint: https://cloud.getdbt.com/api/v2/accounts/{a…

---

## [Can we define vars in dbt-project.yml and utilize that defined vars in dbt-project.yml itself.](https://discourse.getdbt.com/t/can-we-define-vars-in-dbt-project-yml-and-utilize-that-defined-vars-in-dbt-project-yml-itself/20298)

<div class="topic-metadata">

**Author:** [@gayathrikallam3](https://discourse.getdbt.com/u/gayathrikallam3)\
**Replies:** 1\
**Last updated:** [October 8, 2025, 2:37pm UTC](https://discourse.getdbt.com/t/can-we-define-vars-in-dbt-project-yml-and-utilize-that-defined-vars-in-dbt-project-yml-itself/20298 "2025-10-08T14:37:23Z")

</div>

I am trying utilize the vars in dbt-project.yml as a conditional logic for enabling, but getting vars supplied to \[configuration\] = {}. Are we not supposed to utilized vars inside dbt-project.yml The problem I’m havin…

---

## [BigQuery tables or views to expose data?](https://discourse.getdbt.com/t/bigquery-tables-or-views-to-expose-data/20198)

<div class="topic-metadata">

**Author:** [@Etiennr](https://discourse.getdbt.com/u/Etiennr)\
**Replies:** 1\
**Last updated:** [September 8, 2025, 8:00am UTC](https://discourse.getdbt.com/t/bigquery-tables-or-views-to-expose-data/20198 "2025-09-08T08:00:06Z")

</div>

Hey everyone :wave: I am wondering if you use tables or views to expose data in BigQuery, and if so why? 1) Context We use dbt + BigQuery in a data mesh setup, where our data products expose one or multiple public data…

---

## [dbt\_profiler - Trino Support and persisting profile to table](https://discourse.getdbt.com/t/dbt-profiler-trino-support-and-persisting-profile-to-table/20197)

<div class="topic-metadata">

**Author:** [@martin.fedorko](https://discourse.getdbt.com/u/martin.fedorko)\
**Replies:** 0\
**Last updated:** [September 4, 2025, 12:06pm UTC](https://discourse.getdbt.com/t/dbt-profiler-trino-support-and-persisting-profile-to-table/20197 "2025-09-04T12:06:42Z")

</div>

Problem I’m having I write this tread especially for @data-mie, but maybe others will be able to help me too. I have two kinds of problem. First, I would like to create data profile for data objects in dbt with help of …

---

## [Partitions lost during SQL-view (GA4 package)](https://discourse.getdbt.com/t/partitions-lost-during-sql-view-ga4-package/20201)

<div class="topic-metadata">

**Author:** [@bart\_kd](https://discourse.getdbt.com/u/bart_kd)\
**Replies:** 0\
**Last updated:** [September 5, 2025, 2:37pm UTC](https://discourse.getdbt.com/t/partitions-lost-during-sql-view-ga4-package/20201 "2025-09-05T14:37:07Z")

</div>

The problem I’m having I’m using the GA4-package (Velir) in dbt to transform al the GA4-data in my BigQuery Warehouse. However, the view called ‘stg\_ga4\_\_events.sql’ seems unable to use the partitions that are available …

---

## [Concurrent dbt runs](https://discourse.getdbt.com/t/concurrent-dbt-runs/11101)

<div class="topic-metadata">

**Author:** [@siddharth.bose](https://discourse.getdbt.com/u/siddharth.bose)\
**Replies:** 2\
**Last updated:** [August 20, 2025, 3:19pm UTC](https://discourse.getdbt.com/t/concurrent-dbt-runs/11101 "2025-08-20T15:19:10Z")

</div>

We use airflow to orchestrate our dbt runs. Currently we have our entire pipeline run once a day. Though, we have requests to increase the frequency of the runs up to once every 15 mins for parts of the pipeline. My qu…

---

## [\[BUG\] VSCE BigQuery Source Error (Failed to download source schema)](https://discourse.getdbt.com/t/bug-vsce-bigquery-source-error-failed-to-download-source-schema/20152)

<div class="topic-metadata">

**Author:** [@lachiefielding](https://discourse.getdbt.com/u/lachiefielding)\
**Replies:** 0\
**Last updated:** [August 19, 2025, 11:55am UTC](https://discourse.getdbt.com/t/bug-vsce-bigquery-source-error-failed-to-download-source-schema/20152 "2025-08-19T11:55:11Z")

</div>

Trying the new dbt extension in vs code after dbt power user working seemlessly for months. Getting this failed to download source schema error on all of my source schemas, such as the error below. It says it is a big qu…

---

## [BigQuery - adapter.get\_relation() not finding a table that exists](https://discourse.getdbt.com/t/bigquery-adapter-get-relation-not-finding-a-table-that-exists/20120)

<div class="topic-metadata">

**Author:** [@jaboydev](https://discourse.getdbt.com/u/jaboydev)\
**Replies:** 0\
**Last updated:** [August 14, 2025, 2:57am UTC](https://discourse.getdbt.com/t/bigquery-adapter-get-relation-not-finding-a-table-that-exists/20120 "2025-08-14T02:57:23Z")

</div>

The problem I’m having BigQuery adapter.get\_relation() not finding a table. When checking the manifest the project and data locations all match expected, and there are no casing issues. The context of why I’m trying to …

---

## [Use in dbt the bigquery dataset option storage\_billing\_model to PHYSICAL](https://discourse.getdbt.com/t/use-in-dbt-the-bigquery-dataset-option-storage-billing-model-to-physical/11508)

<div class="topic-metadata">

**Author:** [@majorcemp](https://discourse.getdbt.com/u/majorcemp)\
**Replies:** 6\
**Last updated:** [August 8, 2025, 10:24am UTC](https://discourse.getdbt.com/t/use-in-dbt-the-bigquery-dataset-option-storage-billing-model-to-physical/11508 "2025-08-08T10:24:51Z")

</div>

I can’t find the Bigquery dataset option storage\_billing\_model to use PHYSICAL instead of LOGICAL (default) storage of Bigquery when using dbt core (version 0.25) The physical option will be able to lower our cost of st…

---

## [Help with DBT CLI and Dataset Naming in BigQuery](https://discourse.getdbt.com/t/help-with-dbt-cli-and-dataset-naming-in-bigquery/19846)

<div class="topic-metadata">

**Author:** [@raihan](https://discourse.getdbt.com/u/raihan)\
**Replies:** 2\
**Last updated:** [July 14, 2025, 9:50am UTC](https://discourse.getdbt.com/t/help-with-dbt-cli-and-dataset-naming-in-bigquery/19846 "2025-07-14T09:50:57Z")

</div>

The problem I’m having I’m using dbt CLI for data transformations in BigQuery, and I’m encountering an issue with how datasets are named after transformations. When I specify a dataset during project initialization (e.g.…

---

## [dbt Core 1.9 - Intermittent error "OSError: \[Errno 116\] Stale file handle"](https://discourse.getdbt.com/t/dbt-core-1-9-intermittent-error-oserror-errno-116-stale-file-handle/19757)

<div class="topic-metadata">

**Author:** [@paulowiz](https://discourse.getdbt.com/u/paulowiz)\
**Replies:** 0\
**Last updated:** [July 2, 2025, 6:36am UTC](https://discourse.getdbt.com/t/dbt-core-1-9-intermittent-error-oserror-errno-116-stale-file-handle/19757 "2025-07-02T06:36:30Z")

</div>

The problem I’m having We are experiencing intermittent OSError: \[Errno 116\] Stale file handle errors when running dbt commands inside a Google Cloud Run Job. The error occurs during the dbt parsing phase, specifically w…

---

## [Issue with dbt Unit Tests for BigQuery: Handling of ARRAY\<STRING\> Columns in Input Fixtures as CSV](https://discourse.getdbt.com/t/issue-with-dbt-unit-tests-for-bigquery-handling-of-array-string-columns-in-input-fixtures-as-csv/19744)

<div class="topic-metadata">

**Author:** [@amolvetal](https://discourse.getdbt.com/u/amolvetal)\
**Replies:** 0\
**Last updated:** [June 30, 2025, 9:53am UTC](https://discourse.getdbt.com/t/issue-with-dbt-unit-tests-for-bigquery-handling-of-array-string-columns-in-input-fixtures-as-csv/19744 "2025-06-30T09:53:26Z")

</div>

The problem I’m having We have encountered discrepancies between Spark and BigQuery when using dbt unit tests. Specifically, these issues arise when handling input fixture data for columns of type ARRAY. Below is a more …

---

## [Cross-Region Query Error in dbt Cloud with BigQuery](https://discourse.getdbt.com/t/cross-region-query-error-in-dbt-cloud-with-bigquery/19601)

<div class="topic-metadata">

**Author:** [@vigneshmailappan](https://discourse.getdbt.com/u/vigneshmailappan)\
**Replies:** 0\
**Last updated:** [June 4, 2025, 8:16am UTC](https://discourse.getdbt.com/t/cross-region-query-error-in-dbt-cloud-with-bigquery/19601 "2025-06-04T08:16:47Z")

</div>

How to change location dynamically ? or how to query cross region data ? I’m using dbt Cloud with BigQuery as my data warehouse. My dbt Cloud environment is configured to run in the EU multi-region, but I’m trying to us…

---

## [using historic data from a target incremental table as part of a build](https://discourse.getdbt.com/t/using-historic-data-from-a-target-incremental-table-as-part-of-a-build/19375)

<div class="topic-metadata">

**Author:** [@proetman](https://discourse.getdbt.com/u/proetman)\
**Replies:** 1\
**Last updated:** [May 14, 2025, 12:34pm UTC](https://discourse.getdbt.com/t/using-historic-data-from-a-target-incremental-table-as-part-of-a-build/19375 "2025-05-14T12:34:55Z")

</div>

TABLE\_A -------\> TABLE\_B-------\>TARGET (new partition) (historic partition)

---

## [Downstream models still get built even when relationships test fails in dbt build](https://discourse.getdbt.com/t/downstream-models-still-get-built-even-when-relationships-test-fails-in-dbt-build/19360)

<div class="topic-metadata">

**Author:** [@gracia10](https://discourse.getdbt.com/u/gracia10)\
**Replies:** 1\
**Last updated:** [May 13, 2025, 12:42pm UTC](https://discourse.getdbt.com/t/downstream-models-still-get-built-even-when-relationships-test-fails-in-dbt-build/19360 "2025-05-13T12:42:47Z")

</div>

The problem I’m having According to the official docs, when a test on an upstream resource fails during dbt build, downstream models should be skipped (“Skipping on failures”). However, after creating two staging models…

---

## [Help with dbt.this in Incremental Python Models (BigQuery with Hyphen in Project Name)](https://discourse.getdbt.com/t/help-with-dbt-this-in-incremental-python-models-bigquery-with-hyphen-in-project-name/18729)

<div class="topic-metadata">

**Author:** [@linas.kapocius](https://discourse.getdbt.com/u/linas.kapocius)\
**Replies:** 1\
**Last updated:** [March 21, 2025, 8:42am UTC](https://discourse.getdbt.com/t/help-with-dbt-this-in-incremental-python-models-bigquery-with-hyphen-in-project-name/18729 "2025-03-21T08:42:09Z")

</div>

The problem I’m having I am not able to use dbt.this on Python incremental models. The context of why I’m trying to do this I’m trying to implement incremental Python models in dbt, but I’m running into issues when usin…

---

## [data from sql server to bigquery](https://discourse.getdbt.com/t/data-from-sql-server-to-bigquery/8571)

<div class="topic-metadata">

**Author:** [@eamzallag](https://discourse.getdbt.com/u/eamzallag)\
**Replies:** 2\
**Last updated:** [March 20, 2025, 5:57am UTC](https://discourse.getdbt.com/t/data-from-sql-server-to-bigquery/8571 "2025-03-20T05:57:46Z")

</div>

The problem I’m having I would like to upload data from sql server to bigquery The context of why I’m trying to do this transferring data What I’ve already tried I have on core-sqlserver project and core-bigquery proje…

---

## [Connection error dbt to bigquery, 'NoneType' object has no attribute 'close'](https://discourse.getdbt.com/t/connection-error-dbt-to-bigquery-nonetype-object-has-no-attribute-close/18289)

<div class="topic-metadata">

**Author:** [@aziz](https://discourse.getdbt.com/u/aziz)\
**Replies:** 1\
**Last updated:** [March 6, 2025, 3:09am UTC](https://discourse.getdbt.com/t/connection-error-dbt-to-bigquery-nonetype-object-has-no-attribute-close/18289 "2025-03-06T03:09:18Z")

</div>

i tried to connect dbt and bigquery and facing this problem, anyone experienced the same issue? i’m using oauth method for connection

---

## [dbt deps issue in new initialized vscode env](https://discourse.getdbt.com/t/dbt-deps-issue-in-new-initialized-vscode-env/14135)

<div class="topic-metadata">

**Author:** [@sreenig](https://discourse.getdbt.com/u/sreenig)\
**Replies:** 6\
**Last updated:** [February 20, 2025, 6:28pm UTC](https://discourse.getdbt.com/t/dbt-deps-issue-in-new-initialized-vscode-env/14135 "2025-02-20T18:28:28Z")

</div>

Initialized dbt in vscode for the first time after testing my bigquery adapter connection. But when I run dbt deps or dbt --no-version-check deps code, I get the following error PermissionError: \[WinError 32\] The proces…

---

## [Big Query Config block on Incremental merge\_exclude\_columns](https://discourse.getdbt.com/t/big-query-config-block-on-incremental-merge-exclude-columns/18147)

<div class="topic-metadata">

**Author:** [@Karthikeyan](https://discourse.getdbt.com/u/Karthikeyan)\
**Replies:** 0\
**Last updated:** [February 12, 2025, 11:19am UTC](https://discourse.getdbt.com/t/big-query-config-block-on-incremental-merge-exclude-columns/18147 "2025-02-12T11:19:29Z")

</div>

The problem I’m having in Big Query: {{config( materialized=‘incremental’ ,schema= abc ,incremental\_strategy = ‘merge’ ,unique\_key=\[‘EMP\_ID’\] ,merge\_exclude\_columns = \[‘Audit\_ID’, ‘Crt\_Timestamp’\]}} I need to conve…

---

## [installing dbt packages where we have multiple dev and prod data lake, data warehouse, and data mart environments in bigquery](https://discourse.getdbt.com/t/installing-dbt-packages-where-we-have-multiple-dev-and-prod-data-lake-data-warehouse-and-data-mart-environments-in-bigquery/18054)

<div class="topic-metadata">

**Author:** [@bparbhu](https://discourse.getdbt.com/u/bparbhu)\
**Replies:** 0\
**Last updated:** [February 5, 2025, 6:49pm UTC](https://discourse.getdbt.com/t/installing-dbt-packages-where-we-have-multiple-dev-and-prod-data-lake-data-warehouse-and-data-mart-environments-in-bigquery/18054 "2025-02-05T18:49:25Z")

</div>

Right now I’m trying to install dbt packages where we are trying to utilize packages that are sourced from our dev and prod data lake environments and then create stg reports in dev and prod data warehouse and then ultim…

---

## [Snapshots are built with duplicate rows](https://discourse.getdbt.com/t/snapshots-are-built-with-duplicate-rows/9941)

<div class="topic-metadata">

**Author:** [@patryk](https://discourse.getdbt.com/u/patryk)\
**Replies:** 4\
**Last updated:** [February 4, 2025, 10:44pm UTC](https://discourse.getdbt.com/t/snapshots-are-built-with-duplicate-rows/9941 "2025-02-04T22:44:34Z")

</div>

I’m building snapshot models for reverse ETL scripts, so they pickup only recently changed rows. I don’t have any timestamp column available for this and I’m using “check all” strategy. The problem is, two of my snapsho…

---

## [Aggregating multiple bigquery datasets](https://discourse.getdbt.com/t/aggregating-multiple-bigquery-datasets/18019)

<div class="topic-metadata">

**Author:** [@wes](https://discourse.getdbt.com/u/wes)\
**Replies:** 0\
**Last updated:** [February 3, 2025, 7:33pm UTC](https://discourse.getdbt.com/t/aggregating-multiple-bigquery-datasets/18019 "2025-02-03T19:33:40Z")

</div>

Hi! we currently have about 200 datasets that begin with a string ie. analytics-. They all contain the same tables with the same schema. Is it possible to aggregate them all into a single dataset using dbt? Any help woul…

---

## [Do all columns in target have to be in the model](https://discourse.getdbt.com/t/do-all-columns-in-target-have-to-be-in-the-model/17995)

<div class="topic-metadata">

**Author:** [@LiterallyJustAGirl](https://discourse.getdbt.com/u/LiterallyJustAGirl)\
**Replies:** 4\
**Last updated:** [February 3, 2025, 10:21am UTC](https://discourse.getdbt.com/t/do-all-columns-in-target-have-to-be-in-the-model/17995 "2025-02-03T10:21:32Z")

</div>

My model does not contain all the rows that are in my target table, and when i try running it it fails with an unrecognised name error I am trying to make it so that my model runs and populates only the columns in the m…

---

## [Incremental model with no new data](https://discourse.getdbt.com/t/incremental-model-with-no-new-data/17741)

<div class="topic-metadata">

**Author:** [@MartinK](https://discourse.getdbt.com/u/MartinK)\
**Replies:** 1\
**Last updated:** [January 21, 2025, 1:26pm UTC](https://discourse.getdbt.com/t/incremental-model-with-no-new-data/17741 "2025-01-21T13:26:01Z")

</div>

The problem I’m having I have the same problem, which is described here: but, I don’t know how to solve this, the solution description is too brief. Full-refresh of the model and incremental run with new data works fi…

[Next page](https://discourse.getdbt.com/tag/bigquery/36.md?match_all_tags=true&page=1&tags%5B%5D=bigquery)
