# \#python-models

**URL:** https://discourse.getdbt.com/tag/python-models/43.md

[Latest](https://discourse.getdbt.com/latest.md) · [Categories](https://discourse.getdbt.com/categories.md) · [Tags](https://discourse.getdbt.com/tags.md)

---

## [Long delay after writing table from dataframe](https://discourse.getdbt.com/t/long-delay-after-writing-table-from-dataframe/20593)

<div class="topic-metadata">

**Author:** [@chuntuk](https://discourse.getdbt.com/u/chuntuk)\
**Replies:** 0\
**Last updated:** [February 18, 2026, 12:35pm UTC](https://discourse.getdbt.com/t/long-delay-after-writing-table-from-dataframe/20593 "2026-02-18T12:35:21Z")

</div>

I’m having a problem when populating a Databricks table from a Python script that is structured something like this: from pyspark.sql import SparkSession, DataFrame from pyspark.sql.functions import ( col, when, lit…

---

## [I am trying to convert phone numbers to E.164 format](https://discourse.getdbt.com/t/i-am-trying-to-convert-phone-numbers-to-e-164-format/7586)

<div class="topic-metadata">

**Author:** [@sivareddy3244](https://discourse.getdbt.com/u/sivareddy3244)\
**Replies:** 18\
**Last updated:** [November 25, 2025, 1:20pm UTC](https://discourse.getdbt.com/t/i-am-trying-to-convert-phone-numbers-to-e-164-format/7586 "2025-11-25T13:20:23Z")

</div>

The problem I have a table where it consists of two columns with customer phone numbers and Country . I need to convert it to E.164 format . The context of why I’m trying to do this I am trying to solve with the dbt uti…

---

## [dbt python model (using prophet library) on DataProc Serverless cannot connect to pypi](https://discourse.getdbt.com/t/dbt-python-model-using-prophet-library-on-dataproc-serverless-cannot-connect-to-pypi/20393)

<div class="topic-metadata">

**Author:** [@mgloel81](https://discourse.getdbt.com/u/mgloel81)\
**Replies:** 1\
**Last updated:** [November 11, 2025, 9:25am UTC](https://discourse.getdbt.com/t/dbt-python-model-using-prophet-library-on-dataproc-serverless-cannot-connect-to-pypi/20393 "2025-11-11T09:25:19Z")

</div>

The problem I’m having I am trying to run a python model that makes use of facebook’s prophet library to run time series prediction on one of my tables and write the predictions into a new table. The script itself is st…

---

## [Python model warehouse cfg with Databricks](https://discourse.getdbt.com/t/python-model-warehouse-cfg-with-databricks/19311)

<div class="topic-metadata">

**Author:** [@zoraida.hidalgo](https://discourse.getdbt.com/u/zoraida.hidalgo)\
**Replies:** 0\
**Last updated:** [May 8, 2025, 8:34am UTC](https://discourse.getdbt.com/t/python-model-warehouse-cfg-with-databricks/19311 "2025-05-08T08:34:43Z")

</div>

The problem I’m having I have Python models that i want to run with with an All Purpose Compute cluster for staging env and with a Job Compute cluster for production env. I have tried creating a new Connection for the …

---

## [Integrating python with dbt-core](https://discourse.getdbt.com/t/integrating-python-with-dbt-core/18643)

<div class="topic-metadata">

**Author:** [@chakresh.singh](https://discourse.getdbt.com/u/chakresh.singh)\
**Replies:** 0\
**Last updated:** [March 14, 2025, 9:31am UTC](https://discourse.getdbt.com/t/integrating-python-with-dbt-core/18643 "2025-03-14T09:31:15Z")

</div>

Hi all, I have recently began using dbt-core for data processing at our company. When working with raw or evolving datasets, I often find it easier to perform initial data exploration and preprocessing in Python before…

---

## [python models with postgresql warehouse](https://discourse.getdbt.com/t/python-models-with-postgresql-warehouse/7888)

<div class="topic-metadata">

**Author:** [@m.chavoshi96](https://discourse.getdbt.com/u/m.chavoshi96)\
**Replies:** 3\
**Last updated:** [March 8, 2025, 5:47pm UTC](https://discourse.getdbt.com/t/python-models-with-postgresql-warehouse/7888 "2025-03-08T17:47:48Z")

</div>

Can I have python models with a postgresql warehouse and dbt cli?

---

## [Using 'boto3' with Python models on Snowflake](https://discourse.getdbt.com/t/using-boto3-with-python-models-on-snowflake/17200)

<div class="topic-metadata">

**Author:** [@albert](https://discourse.getdbt.com/u/albert)\
**Replies:** 6\
**Last updated:** [December 6, 2024, 2:12am UTC](https://discourse.getdbt.com/t/using-boto3-with-python-models-on-snowflake/17200 "2024-12-06T02:12:00Z")

</div>

The problem I’m having I have boto3 installed in my virtual env but when I run dbt, it cannot find it I am using dbt-core pip install dbt-core dbt-snowflake When running pip freeze | grep dbt I get dbt-adapters==1.10…

---

## [Schema mismatch on dbt python model run](https://discourse.getdbt.com/t/schema-mismatch-on-dbt-python-model-run/11226)

<div class="topic-metadata">

**Author:** [@yekaterina.sun](https://discourse.getdbt.com/u/yekaterina.sun)\
**Replies:** 1\
**Last updated:** [November 29, 2024, 3:10pm UTC](https://discourse.getdbt.com/t/schema-mismatch-on-dbt-python-model-run/11226 "2024-11-29T15:10:20Z")

</div>

Simple dbt python model, 5 fields( 2 INTEGER, 1 TIMESTAMP, 2 STRING). All NULLABLE. The model runs successfully only the first time , creates a BigQuery table, populates it with data. Any subsequent model run causes an…

---

## [Error while using dbutils.jobs.taskValues.set(key="file\_name", value="xyz\_file") statment in dbt python model](https://discourse.getdbt.com/t/error-while-using-dbutils-jobs-taskvalues-set-key-file-name-value-xyz-file-statment-in-dbt-python-model/16559)

<div class="topic-metadata">

**Author:** [@shuja.2006](https://discourse.getdbt.com/u/shuja.2006)\
**Replies:** 0\
**Last updated:** [October 11, 2024, 2:42pm UTC](https://discourse.getdbt.com/t/error-while-using-dbutils-jobs-taskvalues-set-key-file-name-value-xyz-file-statment-in-dbt-python-model/16559 "2024-10-11T14:42:55Z")

</div>

The problem I’m having Error while using dbutils.jobs.taskValues.set(key=“file\_name”, value=“xyz\_file”) statment in dbt python model I am calling my dbt model within databricks job but failing with below error Py4JJa…

---

## [dbt python vscode INVALID\_PARAMETER\_VALUE](https://discourse.getdbt.com/t/dbt-python-vscode-invalid-parameter-value/15110)

<div class="topic-metadata">

**Author:** [@froady](https://discourse.getdbt.com/u/froady)\
**Replies:** 2\
**Last updated:** [August 23, 2024, 2:53pm UTC](https://discourse.getdbt.com/t/dbt-python-vscode-invalid-parameter-value/15110 "2024-08-23T14:53:27Z")

</div>

The problem I’m having I can’t run a python model from my computer. I can run all my SQL models and want to try my first python model. Having trouble finding why I’m getting the error, or good information about how to w…

---

## [DBT Python Model for DBT-Postgres Adapter](https://discourse.getdbt.com/t/dbt-python-model-for-dbt-postgres-adapter/14556)

<div class="topic-metadata">

**Author:** [@panpaliyapratiksha18](https://discourse.getdbt.com/u/panpaliyapratiksha18)\
**Replies:** 4\
**Last updated:** [July 29, 2024, 7:29am UTC](https://discourse.getdbt.com/t/dbt-python-model-for-dbt-postgres-adapter/14556 "2024-07-29T07:29:12Z")

</div>

Not able to create python model in Project which has dbt-postgres and dbt-core getting below error Materialization “materialization\_table\_default” only supports languages \[‘sql’\]; got “python” Library using dbt-core=…

---

## [execute DBT models with python runner dynamically](https://discourse.getdbt.com/t/execute-dbt-models-with-python-runner-dynamically/13323)

<div class="topic-metadata">

**Author:** [@RP\_007](https://discourse.getdbt.com/u/RP_007)\
**Replies:** 1\
**Last updated:** [June 14, 2024, 5:24am UTC](https://discourse.getdbt.com/t/execute-dbt-models-with-python-runner-dynamically/13323 "2024-06-14T05:24:09Z")

</div>

I recently came across python models in DBT and DBTRunner if there any way that we can combine both in one py model? It will run model names to be executed from a table and execute them

---

## [pyspark error when running a python model in athena](https://discourse.getdbt.com/t/pyspark-error-when-running-a-python-model-in-athena/13375)

<div class="topic-metadata">

**Author:** [@ido.horowitz](https://discourse.getdbt.com/u/ido.horowitz)\
**Replies:** 0\
**Last updated:** [May 15, 2024, 1:33pm UTC](https://discourse.getdbt.com/t/pyspark-error-when-running-a-python-model-in-athena/13375 "2024-05-15T13:33:49Z")

</div>

Hey i am a user of dbt core with amazon athena as a db I am trying to run a python model in dbt and run into a pyspark error. The code I am trying to run is really simple (for debugging) , just pointing a ref and tryin…

---

## [How to config to allow new fields when running PySpark Model in GCP DataProc Serverless](https://discourse.getdbt.com/t/how-to-config-to-allow-new-fields-when-running-pyspark-model-in-gcp-dataproc-serverless/9455)

<div class="topic-metadata">

**Author:** [@toan.le](https://discourse.getdbt.com/u/toan.le)\
**Replies:** 3\
**Last updated:** [May 14, 2024, 4:45pm UTC](https://discourse.getdbt.com/t/how-to-config-to-allow-new-fields-when-running-pyspark-model-in-gcp-dataproc-serverless/9455 "2024-05-14T16:45:47Z")

</div>

The problem I’m having When updating my python model to have more fields, PySpark job give warning that the number of fields is mismatched: WARN BigQueryDataSourceWriterInsertableRelation: unexpected issue trying to sav…

---

## [executing python model via dbt terminal](https://discourse.getdbt.com/t/executing-python-model-via-dbt-terminal/13318)

<div class="topic-metadata">

**Author:** [@RP\_007](https://discourse.getdbt.com/u/RP_007)\
**Replies:** 3\
**Last updated:** [May 13, 2024, 8:14am UTC](https://discourse.getdbt.com/t/executing-python-model-via-dbt-terminal/13318 "2024-05-13T08:14:25Z")

</div>

The problem I’m having I want to execute python model created in dbt core The context of why I’m trying to do this I created a sample file with name my\_first\_python as below: import pandas as pd def model(dbt,session)…

---

## [What is the roadmap for Python models?](https://discourse.getdbt.com/t/what-is-the-roadmap-for-python-models/10189)

<div class="topic-metadata">

**Author:** [@keller](https://discourse.getdbt.com/u/keller)\
**Replies:** 1\
**Last updated:** [May 10, 2024, 6:22am UTC](https://discourse.getdbt.com/t/what-is-the-roadmap-for-python-models/10189 "2024-05-10T06:22:56Z")

</div>

I am aware of the limitations of Python models in dbt and the pros and cons in comparison to SQL models. However, the main draw to our team of using dbt is that our core DE team is Python-first, and we also have a number…

---

## [Need help using incremental to append new campaigns to the table and incrementing surrogate key by 1](https://discourse.getdbt.com/t/need-help-using-incremental-to-append-new-campaigns-to-the-table-and-incrementing-surrogate-key-by-1/13145)

<div class="topic-metadata">

**Author:** [@dchen](https://discourse.getdbt.com/u/dchen)\
**Replies:** 1\
**Last updated:** [May 1, 2024, 5:29pm UTC](https://discourse.getdbt.com/t/need-help-using-incremental-to-append-new-campaigns-to-the-table-and-incrementing-surrogate-key-by-1/13145 "2024-05-01T17:29:14Z")

</div>

I need help in creating my first incremental model. In the campaignchannel table, I have a surrogate\_key column which takes the clientcode\_id and concats the index number to create it, which acts like an unique key. The…

---

## [Declare Column Data Types in python models](https://discourse.getdbt.com/t/declare-column-data-types-in-python-models/13082)

<div class="topic-metadata">

**Author:** [@mra](https://discourse.getdbt.com/u/mra)\
**Replies:** 0\
**Last updated:** [April 25, 2024, 9:06pm UTC](https://discourse.getdbt.com/t/declare-column-data-types-in-python-models/13082 "2024-04-25T21:06:11Z")

</div>

In my python model I have created a data frame with 3 columns, 2 string columns and one is json type. When I execute the model, it runs without any error but in my destination big query table , all of the columns are se…

---

## [dbt python dynamic passing of table not working](https://discourse.getdbt.com/t/dbt-python-dynamic-passing-of-table-not-working/12590)

<div class="topic-metadata">

**Author:** [@ash1595vkc](https://discourse.getdbt.com/u/ash1595vkc)\
**Replies:** 0\
**Last updated:** [March 22, 2024, 9:55pm UTC](https://discourse.getdbt.com/t/dbt-python-dynamic-passing-of-table-not-working/12590 "2024-03-22T21:55:14Z")

</div>

The problem I’m having is to dynamically pass the test names to dbt.ref and dbt .source() " The context of why I’m trying to do this im trying to build a data frame with test name and test results wrapped in json format…

---

## [Network configuration to run dbt Python models using Dataproc Cluster](https://discourse.getdbt.com/t/network-configuration-to-run-dbt-python-models-using-dataproc-cluster/12221)

<div class="topic-metadata">

**Author:** [@alexchouraki](https://discourse.getdbt.com/u/alexchouraki)\
**Replies:** 1\
**Last updated:** [February 27, 2024, 4:07pm UTC](https://discourse.getdbt.com/t/network-configuration-to-run-dbt-python-models-using-dataproc-cluster/12221 "2024-02-27T16:07:38Z")

</div>

The problem I’m having I’m encountering network related errors when running Python models in dbt using a Dataproc cluster. Namely, a 404 error : “The resource ‘projects/analytics-euw4-01/global/networks/default’ was no…

---

## [Can I call a python function inside the macro written in jinja?](https://discourse.getdbt.com/t/can-i-call-a-python-function-inside-the-macro-written-in-jinja/11989)

<div class="topic-metadata">

**Author:** [@pradeep244](https://discourse.getdbt.com/u/pradeep244)\
**Replies:** 3\
**Last updated:** [February 12, 2024, 8:16am UTC](https://discourse.getdbt.com/t/can-i-call-a-python-function-inside-the-macro-written-in-jinja/11989 "2024-02-12T08:16:13Z")

</div>

I have a column which contains a json string. using run\_query() , I captured the json string into a variable inside a macro written in jinja sql. I want to write a python function and call the function inside th…

---

## [Self reference in python model?](https://discourse.getdbt.com/t/self-reference-in-python-model/11423)

<div class="topic-metadata">

**Author:** [@yadderace](https://discourse.getdbt.com/u/yadderace)\
**Replies:** 0\
**Last updated:** [January 5, 2024, 4:55pm UTC](https://discourse.getdbt.com/t/self-reference-in-python-model/11423 "2024-01-05T16:55:44Z")

</div>

The problem I’m having i can’t get the dataframe of the self python model. The context of why I’m trying to do this I need to compare if a record is already stored in order to make another calculations. What I’ve alre…

---

## [Run python functions in dbt dag](https://discourse.getdbt.com/t/run-python-functions-in-dbt-dag/10819)

<div class="topic-metadata">

**Author:** [@ajmal92](https://discourse.getdbt.com/u/ajmal92)\
**Replies:** 0\
**Last updated:** [November 16, 2023, 9:21am UTC](https://discourse.getdbt.com/t/run-python-functions-in-dbt-dag/10819 "2023-11-16T09:21:17Z")

</div>

The problem I’m having I have some python scripts that need to be run after a model x has been refreshed. after this step has been completed. This is typically run at the end of raw refresh \[ie, after all the tables with…

---

## [Error running DBt python model](https://discourse.getdbt.com/t/error-running-dbt-python-model/10674)

<div class="topic-metadata">

**Author:** [@akhildevvr](https://discourse.getdbt.com/u/akhildevvr)\
**Replies:** 0\
**Last updated:** [November 7, 2023, 3:11pm UTC](https://discourse.getdbt.com/t/error-running-dbt-python-model/10674 "2023-11-07T15:11:32Z")

</div>

The problem I’m having I am trying to run a python dbt model on version 1.6.6 with Snowflake Plugin version 1.6.4. However i am getting the below error. Some example code or error messages import snowflake.snowpark.fun…

---

## [Python models materialized as Dynamic Tables](https://discourse.getdbt.com/t/python-models-materialized-as-dynamic-tables/10646)

<div class="topic-metadata">

**Author:** [@andrey.fedorov](https://discourse.getdbt.com/u/andrey.fedorov)\
**Replies:** 0\
**Last updated:** [November 5, 2023, 4:30pm UTC](https://discourse.getdbt.com/t/python-models-materialized-as-dynamic-tables/10646 "2023-11-05T16:30:03Z")

</div>

I noticed that Python models are currently documented as only supporting Table and Incremental materializations and the docs make it sound like even custom materializations aren’t supported. Is that the case, and are the…

---

## [Installing python package in DBT](https://discourse.getdbt.com/t/installing-python-package-in-dbt/10329)

<div class="topic-metadata">

**Author:** [@promediazadrianlearn](https://discourse.getdbt.com/u/promediazadrianlearn)\
**Replies:** 1\
**Last updated:** [October 13, 2023, 6:21pm UTC](https://discourse.getdbt.com/t/installing-python-package-in-dbt/10329 "2023-10-13T18:21:00Z")

</div>

The problem I’m having I’m building a model using python and want to install CBSODATA package. I have been searching the tutorial how install the python packages in dbt but could not find one. Can anybody tell me how to …

---

## [Python model running in Snowflake and IDE but not as dbtCloud job](https://discourse.getdbt.com/t/python-model-running-in-snowflake-and-ide-but-not-as-dbtcloud-job/9573)

<div class="topic-metadata">

**Author:** [@grant.beasley](https://discourse.getdbt.com/u/grant.beasley)\
**Replies:** 2\
**Last updated:** [August 24, 2023, 8:49am UTC](https://discourse.getdbt.com/t/python-model-running-in-snowflake-and-ide-but-not-as-dbtcloud-job/9573 "2023-08-24T08:49:38Z")

</div>

I’m trying to run a python model to do a data transformation that is much easier with Pandas than SQL. When I run the python code in Snowflake, the Snowpark dataframe is returned properly. When I run the model as a dbt m…

---

## [Schema error on python-model](https://discourse.getdbt.com/t/schema-error-on-python-model/9359)

<div class="topic-metadata">

**Author:** [@dutenrapha](https://discourse.getdbt.com/u/dutenrapha)\
**Replies:** 0\
**Last updated:** [July 31, 2023, 8:33pm UTC](https://discourse.getdbt.com/t/schema-error-on-python-model/9359 "2023-07-31T20:33:29Z")

</div>

Hey everyone, I’m having a weird issue with dbt. I created this very simple model /model.py import pandas as pd def model(dbt, session): dbt.config(materialized = "table", packages = \["pandas"\]) df = dbt.ref(…

---

## [What's the Jinja (dbt-utils) alternative of dbt-Python models?](https://discourse.getdbt.com/t/whats-the-jinja-dbt-utils-alternative-of-dbt-python-models/9275)

<div class="topic-metadata">

**Author:** [@markcruz](https://discourse.getdbt.com/u/markcruz)\
**Replies:** 0\
**Last updated:** [July 24, 2023, 10:11pm UTC](https://discourse.getdbt.com/t/whats-the-jinja-dbt-utils-alternative-of-dbt-python-models/9275 "2023-07-24T22:11:41Z")

</div>

The problem I’m having How do I go about using dbt-utils macros within Python models? Is this even possible? What is the alternative? The context of why I’m trying to do this I would like to reuse the macros already imp…

---

## ["Need to supply dataproc\_region in profile to submit python job" Although it is already supplied.](https://discourse.getdbt.com/t/need-to-supply-dataproc-region-in-profile-to-submit-python-job-although-it-is-already-supplied/8572)

<div class="topic-metadata">

**Author:** [@candassahin](https://discourse.getdbt.com/u/candassahin)\
**Replies:** 0\
**Last updated:** [June 5, 2023, 1:36pm UTC](https://discourse.getdbt.com/t/need-to-supply-dataproc-region-in-profile-to-submit-python-job-although-it-is-already-supplied/8572 "2023-06-05T13:36:09Z")

</div>

I am using dbt Cloud to deploy. I had a single job that runs everyday. After 26th of May, I started to get “Need to supply dataproc\_region in profile to submit python job” error while running my python based model. It i…

[Next page](https://discourse.getdbt.com/tag/python-models/43.md?match_all_tags=true&page=1&tags%5B%5D=python-models)