dbt with AWS Athena and Python models.

From Python models | dbt Developer Hub , seems only the following 3 platforms supports Python models.

  • Databricks, Snowflakes, BigQuery.

I am thinking to have dbt get source tables from Athena, doing some transform, then store the result in S3. However, I cannot find a way if I can use dbt-python models on this? Or it must involve one of above 3 platforms.

Any examples would be super helpful.


Hi Yanan,

At the moment, those three platforms are the only ones that support dbt Python models natively.

However, you might be able to use the fal-dbt adapter to run your Python workloads.


Thanks Joel for the help. I did a quick try on dbt-fal, by following the link:

  1. The profile seems file, with fal in the picture:
        target: dev
            type: fal
            db_profile: dev_athena
            type: athena
  1. I have a very simple sql and py file, the py ref to the sql:
      import pandas
      def model(dbt, fal):
          df = dbt.ref("test_query")
          return df
  1. dbt run got the success on sql, but not the py:
       $ dbt run
        19:14:51  Running with dbt=1.3.2
        19:14:51  Found 2 models, 0 tests, 0 snapshots, 0 analyses, 282 macros, 0 operations, 0 seed files, 6 sources, 0 exposures, 0 metrics
        19:14:55  Concurrency: 1 threads (target='dev')
        19:14:55  1 of 2 START sql table model dbt.test_query .................................... [RUN]
        19:15:14  1 of 2 OK created sql table model dbt.test_query ............................... [**OK -1 in 18.75s**]
        19:15:14  2 of 2 START python view model dbt.demo_athena_python .......................... [RUN]
        19:15:14  2 of 2 ERROR creating python view model dbt.demo_athena_python ................. [**ERROR in 0.03s**]
        19:15:14  Finished running 1 table model, 1 view model in 0 hours 0 minutes and 23.07 seconds (23.07s).
        19:15:14  Completed with 1 error and 0 warnings:
        19:15:14  Runtime Error in model demo_athena_python (models\demo_athena_python.py)
        19:15:14    **Materialization "materialization_view_athena" only supports languages ['sql']; got "python"**
        19:15:14  Done. PASS=1 WARN=0 ERROR=1 SKIP=0 TOTAL=2

feels like it still may not work on my scenario ‘athena’ case. Or something I missed?

Hey there, this is Meder from fal. We indeed do not support Athena yet (thanks for pointers @joellabes). I just created a ticket for it, so we should get it working pretty soon. In the meantime, @yzhang, if you have time, you can create an issue in our github repository: GitHub - fal-ai/fal: do more with dbt. fal helps you run Python alongside dbt, so you can send Slack alerts, detect anomalies and build machine learning models.


Hi again! So we just released dbt-fal 1.3.13 that enables support for dbt-athena-community. To install, run pip install dbt-fal==1.3.13. @yzhang, your Python models should compute as long as you set materialized config to table:

import pandas
def model(dbt, fal):
  df = dbt.ref("test_query")
  return df

Let me know if it works!


Thank you! I just confirmed, with dbt.config(materialized=“table” , it went through my code. Again, thank you very much for the help.


This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.