Python model doesn't create table in BigQuery

Help
1

I’m trying to create two new columns from an intermediate table into a final table in BigQuery.
I’m using Pyspark; everything runs without any errors, but the table is not created in the end.

Here is my model, the two functions that I’m calling are UDFs.

def model(dbt, session):
    dbt.config(materialized="table", tags=["monitoring"])
    sequence_matcher_ratio_threshold = dbt.config.get("sequence_matcher_ratio_threshold")
    df = dbt.ref("int_messages_monitoring")
    df = df.withColumn(
        "sequence_matcher_ratio",
        get_sequence_matcher_ratio("sample_message", "chat_messages")
    )
    df = df.withColumn(
        "message_suggested_outcome",
        apply_threshold_on_message_suggested_outcome(
            "message_suggested_outcome_without_threshold",
            "sequence_matcher_ratio",
            lit(sequence_matcher_ratio_threshold)
        )
    )
    return df
1 Like